Research on image steganography analysis based on deep learning
تحقیق در مورد تجزیه و تحلیل استگانوگرافی تصویر بر اساس یادگیری عمیق-2019
Although steganalysis has developed rapidly in recent years, it still faces many difficulties and challenges. Based on the theory of in-depth learning method and image-based general steganalysis, this paper makes a deep study of the hot and difficult problem of steganalysis feature expression, and tries to establish a new steganalysis paradigm from the idea of feature learning. The main contributions of this paper are as follows: 1. An innovative steganalysis paradigm based on in-depth learning is proposed. Based on the representative deep learning method CNN, the model is designed and adjusted according to the characteristics of steganalysis, which makes the proposed model more effective in capturing the statistical characteristics such as neighborhood correlation. 2. A steganalysis feature learning method based on global information constraints is proposed. Based on the previous research of steganalysis method based on CNN, this work focuses on the importance of global information in steganalysis feature expression. 3. A feature learning method for low embedding rate steganalysis is proposed. 4. A general steganalysis method for multi-class steganography is proposed. The ultimate goal of general steganalysis is to construct steganalysis detectors without distinguishing specific types of steganalysis algorithms
Keywords: Steganalysis | Steganography | Feature learning | Deep learning | Convolutional neural network | Transfer learning | Multitask learning
Intelligent fault diagnosis of cooling radiator based on deep learning analysis of infrared thermal images
تشخیص خطای هوشمند رادیاتور خنک کننده بر اساس تجزیه و تحلیل یادگیری عمیق از تصاویر حرارتی مادون قرمز-2019
Detection of faults and intelligent monitoring of equipment operations are essential for modern industries. Cooling radiator condition is one of the factors that affects engine performance. This paper proposes a novel and accurate radiator condition monitoring and intelligent fault detection based on thermal images and using a deep convolutional neural network (CNN) which has a specific configuration to combine the feature extraction and classification steps. The CNN model is constructed from VGG-16 structure that is followed by batch normalization layer, dropout layer, and dense layer. The suggested CNN model directly uses infrared thermal images as input to classify six conditions of the radiator: normal, tubes blockage, coolant leakage, cap failure, loose connections between fins & tubes and fins blockage. Evaluation of the model demonstrates that leads to results better than traditional computational intelligence methods, such as an artificial neural network, and can be employed with high performance and accuracy for fault diagnosis and condition monitoring of the cooling radiator under various working circumstances.
Keywords: Cooling radiator | Fault detection | Thermal image analysis | Deep learning | Convolutional neural network
A deep learning framework for automatic diagnosis of unipolar depression
یک چارچوب یادگیری عمیق برای تشخیص خودکار افسردگی تک قطبی-2019
Background and purpose: In recent years, the development of machine learning (ML) frameworks for automatic diagnosis of unipolar depression has escalated to a next level of deep learning frameworks. However, this idea needs further validation. Therefore, this paper has proposed an electroencephalographic (EEG)-based deep learning framework that automatically discriminated depressed and healthy controls and provided the diagnosis. Basic procedures: In this paper, two different deep learning architectures were proposed that utilized one dimensional convolutional neural network (1DCNN) and 1DCNN with long short-term memory (LSTM) architecture. The proposed deep learning architectures automatically learn patterns in the EEG data that were useful for classifying the depressed and healthy controls. In addition, the proposed models were validated with restingstate EEG data obtained from 33 depressed patients and 30 healthy controls. Main findings: As results, significant differences were observed between the two groups. The classification results involving the CNN model were accuracy=98.32%, precision=99.78%, recall=98.34%, and f-score= 97.65%. In addition, the study has reported LSTM with 1DCNN classification accuracy=95.97%, precision= 99.23%, recall=93.67%, and f-score=95.14%. Conclusions: Deep learning frameworks could revolutionize the clinical applications for EEG-based diagnosis for depression. Based on the results, it may be concluded that the deep learning framework could be used as an automatic method for diagnosing the depression.
Keywords: EEG-based deep learning for depression | EEG-based diagnosis of unipolar depression | Convolutional neural network for depression | Long short-term memory classifiers for depression | EEG-based machine learning methods for depression
Deep convolutional learning for general early design stage prediction models
یادگیری همگرای عمیق برای مدل های پیش گویی مرحله اولیه طراحی-2019
Designers rely on performance predictions to direct the design toward appropriate requirements. Machine learning (ML) models exhibit the potential for rapid and accurate predictions. Developing conventional ML models that can be generalized well in unseen design cases requires an effective feature engineering and selection. Identifying generalizable features calls for good domain knowledge by the ML model developer. Therefore, developing ML models for all design performance parameters with conventional ML will be a timeconsuming and expensive process. Automation in terms of feature engineering and selection will accelerate the use of ML models in design. Deep learning models extract features from data, which aid in model generalization. In this study, we (1) evaluate the deep learning model’s capability to predict the heating and cooling demand on unseen design cases and (2) obtain an understanding of extracted features. Results indicate that deep learning model generalization is similar to or better than that of a simple neural network with appropriate features. The reason for the satisfactory generalization using the deep learning model is its ability to identify similar design options within the data distribution. The results also indicate that deep learning models can filter out irrelevant features, reducing the need for feature selection.
Keywords: Convolutional neural network | Energy predictions | Machine learning | Feature learning
Automated diagnosis of ear disease using ensemble deep learning with a big otoendoscopy image database
تشخیص خودکار بیماری گوش با استفاده از یادگیری عمیق گروه با یک پایگاه داده بزرگ تصویر otoendoscopy-2019
Background: Ear and mastoid disease can easily be treated by early detection and appropriate medical care. However, short of specialists and relatively lowdiagnostic accuracy calls for a newway of diagnostic strategy, inwhich deep learning may play a significant role. The current study presents a machine learning model to automatically diagnose ear disease using a large database of otoendoscopic images acquired in the clinical environment. Methods: Total 10,544 otoendoscopic images were used to train nine public convolution-based deep neural networks to classify eardrum and external auditory canal features into six categories of ear diseases, covering most ear diseases (Normal, Attic retraction, Tympanic perforation, Otitis externa±myringitis, Tumor). After evaluating several optimization schemes, two best-performingmodelswere selected to compose an ensemble classifier, by combining classification scores of each classifier. Findings: According to accuracy and training time, transfer learning models based on Inception-V3 and ResNet101 were chosen and the ensemble classifier using the two models yielded a significant improvement over each model, the accuracy of which is in average 93·67% for the 5-folds cross-validation. Considering substantial data-size dependency of classifier performance in the transfer learning, evaluated in this study, the high accuracy in the current model is attributable to the large database. Interpretation: The current study is unprecedented in terms of both disease diversity and diagnostic accuracy, which is compatible or even better than an average otolaryngologist. The classifier was trainedwith data in a various acquisition condition,which is suitable for the practical environment. This study shows the usefulness of utilizing a deep learning model in the early detection and treatment of ear disease in the clinical situation. Fund: This research was supported by Brain Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT(NRF-2017M3C7A1049051).
Keywords: Convolutional neural network | Deep learning | Otoendoscopy | Tympanic membrane | Ear disease | Ensemble learning
Deep learning for vibrational spectral analysis: Recent progress and a practical guide
یادگیری عمیق برای تجزیه و تحلیل طیفی ارتعاش: پیشرفت های اخیر و یک راهنمای عملی-2019
The development of chemometrics aims to provide an effective analysis approach for data generated by advanced analytical instruments. The success of existing analytical approaches in spectral analysis still relies on preprocessing and feature selection techniques to remove signal artifacts based on prior experiences. Data-driven deep learning analysis has been developed and successfully applied in many domains in the last few years. How to integrate deep learning with spectral analysis received increased attention for chemometrics. Approximately 20 recently published studies demonstrate that deep neural networks can learn critical patterns from raw spectra, which significantly reduces the demand for feature engineering. The composition of multiple processing layers improves the fitting and feature extraction capability and makes them applicable to various analytical tasks. This advance offers a new solution for chemometrics toward resolving challenges related to spectral data with rapidly increased sample numbers from various sources. We further provide a practical guide to the development of a deep convolutional neural network-based analytical workflow. The design of the network structure, tuning the hyperparameters in the training process, and repeatability of results is mainly discussed. Future studies are needed on interpretability and repeatability of the deep learning approach in spectral analysis.
Keywords: Chemometrics | Artificial intelligence | Deep learning | Spectroscopy | Convolutional neural network | Analysis
Deep learning for the classification of human sperm
یادگیری عمیق برای طبقه بندی اسپرم های انسانی-2019
Background: Infertility is a global health concern, and couples are increasingly seeking medical assistance to achieve reproduction. Semen analysis is a primary assessment performed by a clinician, in which the morphology of the sperm population is evaluated. Machine learning algorithms that automate, standardize, and expedite sperm classification are the subject of ongoing research. Method: We demonstrate a deep learning method to classify sperm into one of several World Health Organization (WHO) shape-based categories. Our method uses VGG16, a deep convolutional neural network (CNN) initially trained on ImageNet, a collection of human-annotated everyday images, which we retrain for sperm classification using two freely-available sperm head datasets (HuSHeM and SCIAN). Results: Our deep learning approach classifies sperm at high accuracy and performs well in head-to-head comparisons with earlier approaches using identical datasets. We demonstrate improvement in true positive rate over a classifier approach based on a cascade ensemble of support vector machines (CE-SVM) and show similar true positive rates as compared to an adaptive patch-based dictionary learning (APDL) method. Retraining an off-the-shelf VGG16 network avoids excessive neural network computation or having to learn and use the massive dictionaries required for sparse representation, both of which can be computationally expensive. Conclusions: We show that our deep learning approach to sperm head classification represents a viable method to automate, standardize, and accelerate semen analysis. Our approach highlights the potential of artificial intelligence technologies to eventually exceed human experts in terms of accuracy, reliability, and throughput.
Keywords: Convolutional neural network | Deep learning | Fertility | Sperm selection | Sperm diagnostics | Sperm head classification | Transfer learning
Evaluating adipocyte differentiation of bone marrow-derived mesenchymal stem cells by a deep learning method for automatic lipid droplet counting
بررسی تمایز چربی سلولهای بنیادی مزانشیمی مشتق از مغز استخوان با استفاده از روش یادگیری عمیق برای شمارش قطرات لیپیدی خودکار-2019
Stem cells are a group of competent cells capable of self-renewal and differentiating into osteogenic, chondrogenic, and adipogenic lineages. These cells provide the possibility of successfully treating patients. During differentiation into adipose tissues, a large number of lipid droplets normally accumulate in these cells, which can be seen through oil red O staining. Although the oil red O staining technique is regularly used for assessing the differentiation degree, its validity for quantitative studies has not been approved yet. Lipid droplet counting has applications in differentiation works and saves time and costs once being automated. In this research, for proving the differentiation of mesenchymal stem cells (MSCs) into adipocyte tissues, their microscopic images were provided. Then, the microscopic images were segmented into square patches, and the lipid droplets were annotated through single-point annotation. The proposed network, based on deep learning, is a fully convolutional regression network processing an image with a small respective field on it. Finally, this method not only does count the lipid droplets but also generates a count map. The average counting accuracy is 94%, which is higher than that of the state-of-the-art methods. It is useful to cell biologists to check the percentage of differentiation in different samples. Also, with a count map, it is possible to observe the regions with high concentrations of lipid droplets without oil red O staining and, thus, examine the total adipocyte differentiation. The contribution of this paper is that a deep learning algorithm has been used for the first time in the field of processing intracellular images.
Keywords: Stem cells | Adipocyte differentiation | Counting | Deep learning | Convolutional neural network | Lipid droplets | Regression
Deep learning based winter wheat mapping using statistical data as ground references in Kansas and northern Texas, US
نقشه برداری گندم زمستانی مبتنی بر یادگیری عمیق با استفاده از داده های آماری به عنوان منابع زمینی در کانزاس و شمال تگزاس ، ایالات متحده-2019
Winter wheat is a major staple crop and it is critical to monitor winter wheat production using efficient and automated means. This study proposed a novel approach to produce winter wheat maps using statistics as the training targets of supervised classification. Deep neural network architectures were built to link remotely sensed image series to the harvested areas of individual administrative units. After training, the resultant maps were generated using the activations on a middle layer of the deep model. The direct use of statistical data to some extent alleviates the shortage of ground samples in classification tasks and provides an opportunity to utilize a wealth of statistical records to improve land use mapping. The experiments were carried out in Kansas and Northern Texas during 2001–2017. For each study area the goal was to create winter maps that are consistent with USDA county-level statistics of harvested areas. The trained deep models automatically identified the seasonal pattern of winter wheat pixels without using pixel-level reference data. The winter wheat maps were compared with the Cropland Data Layer (CDL) for years when the CDL is available. In Kansas where the winter wheat extent of the CDL has high reported accuracy and agrees well with county statistics, the maps produced from the deep model was evaluated using the CDL as an independent test set. Northern Texas was selected as an example where the winter wheat area of the CDL is very different from official statistics, and the maps by the deep model enabled a map-to-map comparison with the CDL to highlight the areas of discrepancy. Visual representation of the deep model behaviors and recognized patterns show that deep learning is an automated and robust means to handle the variability of winter wheat seasonality without the need of manual feature engineering and intensive ground data collection. Showing the possibility of generating maps solely from regional statistics, the proposed deep learning approach has great potential to fill the historical gaps of conventional sample-based classification and extend applications to areas where only regional statistics are available. The flexible deep network architecture can be fused with various statistical datasets to fully employ existing sources of data and knowledge.
Keywords: Crop classification | Deep learning | Artificial neural network | Convolutional neural network | MODIS | Winter wheat | USDA Quick Stats
Deep learning approaches for automatic detection of sleep apnea events from an electrocardiogram
رویکردهای یادگیری عمیق برای تشخیص خودکار رویدادهای sleep apnea از الکتروکاردیوگرام-2019
Background and Objective: This study demonstrates deep learning approaches with an aim to find the optimal method to automatically detect sleep apnea (SA) events from an electrocardiogram (ECG) signal. Methods: Six deep learning approaches were designed and implemented for automatic detection of SA events including deep neural network (DNN), one-dimensional (1D) convolutional neural networks (CNN), two-dimensional (2D) CNN, recurrent neural networks (RNN), long short-term memory, and gated- recurrent unit (GRU). Designed deep learning models were analyzed and compared in the performances. The ECG signal was pre-processed, normalized, and segmented into 10 s intervals. Subsequently, the sig- nal was converted into a 2D form for analysis in the 2D CNN model. A dataset collected from 86 patients with SA was used. The training set comprised data from 69 of the patients, while the test set contained data from the remaining 17 patients. Results: The accuracy of the best-performing model was 99.0%, and the 1D CNN and GRU models had 99.0% recall rates. Conclusions: The designed deep learning approaches performed better than those developed and tested in previous studies in terms of detecting SA events, and they could distinguish between apnea and hypopnea events using an ECG signal. The deep learning approaches such as 1D CNN and GRU can be helpful tools to automatically detect SA in sleep apnea screening and related studies.
Keywords: Sleep apnea | Deep learning | Convolutional neural network | Recurrent neural network | Long short-term memory | Gated-recurrent unit