Image anomaly detection for IoT equipment based on deep learning
تشخیص ناهنجاری تصویر برای تجهیزات اینترنت اشیا بر اساس یادگیری عمیق-2019
Intelligent power grid systems is the trend of power development, since traditional methods of manually monitoring power equipment have been unable to meet the requirements of power systems. When an abnormal situation occurs in the operating environment, most monitoring devices cannot be quickly and accurately identified, which may have serious consequences. Aiming at the above problems, in this paper, we propose an anomaly detection algorithm for the monitoring environment of power IoT equipment operating environment based on deep learning from the perspective of personnel identification and fire smoke detection. The multi-stream CNN-based remote monitoring image personnel detection method and the deep convolutional neural network-based fire smoke detection method have achieved good results in personnel identification and fire smoke detection in the power equipment operating environment monitoring image, respectively. This provides a reference for monitoring image anomaly detection.
Keywords: Operating environment monitoring | Image anomaly detection | Deep learning
Attack and anomaly detection in IoT sensors in IoT sites using machine learning approaches
حمله و تشخیص ناهنجاری در سنسورهای IoT در سایت های IoT با استفاده از روشهای یادگیری ماشین-2019
Attack and anomaly detection in the Internet of Things (IoT) infrastructure is a rising concern in the domain of IoT. With the increased use of IoT infrastructure in every do- main, threats and attacks in these infrastructures are also growing commensurately. De- nial of Service, Data Type Probing, Malicious Control, Malicious Operation, Scan, Spying and Wrong Setup are such attacks and anomalies which can cause an IoT system failure. In this paper, performances of several machine learning models have been compared to predict attacks and anomalies on the IoT systems accurately. The machine learning (ML) algorithms that have been used here are Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), and Artificial Neural Network (ANN). The evaluation metrics used in the comparison of performance are accuracy, precision, recall, f1 score, and area under the Receiver Operating Characteristic Curve. The system obtained 99.4% test accuracy for Decision Tree, Random Forest, and ANN. Though these techniques have the same accuracy, other metrics prove that Random Forest performs comparatively better.
Keywords: Internet of Things (IoT) | Machine Learning | Cybersecurity | Anomaly detection
Deep learning approach for sustainable WWTP operation: A case study on data-driven influent conditions monitoring
رویکرد یادگیری عمیق برای عملکرد پایدار WWTP: مطالعه موردی در زمینه نظارت بر شرایط تأثیرگذار محور داده-2019
Wastewater treatment plants (WWTPs) are sustainable solutions to water scarcity. As initial conditions offered to WWTPs, influent conditions (ICs) affect treatment units states, ongoing processes mechanisms, and product qualities. Anomalies in ICs, often raised by abnormal events, need to be monitored and detected promptly to improve system resilience and provide smart environments. This paper proposed and verified data-driven anomaly detection approaches based on deep learning methods and clustering algorithms. Combining both the ability to capture temporal auto-correlation features among multivariate time series from recurrent neural networks (RNNs), and the function to delineate complex distributions from restricted Boltzmann machines (RBM), RNN-RBM models were employed and connected with various classifiers for anomaly detection. The effectiveness of RNN based, RBM based, RNN-RBM based, or standalone individual detectors, including expectation maximization clustering, K-means clustering, mean-shift clustering, one-class support vector machine (OCSVM), spectral clustering, and agglomerative clustering algorithms were evaluated by importing seven years ICs data from a coastal municipal WWTP where more than 150 abnormal events occurred. Results demonstrated that RNN-RBM-based OCSVM approach outperformed all other scenarios with an area under the curve value up to 0.98, which validated the superiority in feature extraction by RNN-RBM, and the robustness in multivariate nonlinear kernels by OCSVM. The model was flexible for not requiring assumptions on data distribution, and could be shared and transferred among environmental data scientists.
Keywords: Wastewater treatment plant | Influent conditions monitoring | Machine learning | Unsupervised deep learning
Deep learning only by normal brain PET identify unheralded brain anomalies
یادگیری عمیق فقط با PET مغز نرمال ناهنجاریهای مغزی هدایت نشده را شناسایی می کند-2019
Background: Recent deep learning models have shown remarkable accuracy for the diagnostic classification. However, they have limitations in clinical application due to the gap between the training cohorts and realworld data. We aimed to develop a model trained only by normal brain PET data with an unsupervised manner to identify an abnormality in various disorders as imaging data of the clinical routine. Methods: Using variational autoencoder, a type of unsupervised learning, Abnormality Scorewas defined as how far a given brain image is from the normal data. The model was applied to FDG PET data of Alzheimers disease (AD) andmild cognitive impairment (MCI) and clinical routine FDG PET data for assessing behavioral abnormality and seizures. Accuracy was measured by the area under curve (AUC) of receiver-operating-characteristic (ROC) curve.We investigated whether deep learning has additional benefits with experts visual interpretation to identify abnormal patterns. Findings: The AUC of the ROC curve for differentiating AD was 0.90. The changes in cognitive scores frombaseline to 2-year follow-up were significantly correlated with Abnormality Score at baseline. The AUC of the ROC curve for discriminating patients with various disorders from controls was 0.74. Experts visual interpretation was helped by the deep learning model to identify abnormal patterns in 60% of cases initially not identified without the model. Interpretation:We suggest that deep learning model trained only by normal data was applicable for identifying wide-range of abnormalities in brain diseases, even uncommon ones, proposing its possible use for interpreting real-world clinical data.
Keywords: PET | Deep learning | Variational autoencoder | Alzheimer | Anomaly detection
Design of machine learning models with domain experts for automated sensor selection for energy fault detection
طراحی مدلهای یادگیری ماشینی با کارشناسان دامنه برای انتخاب سنسور خودکار برای تشخیص خطای انرژی-2019
Data-driven techniques that extract insights from sensor data reduce the cost of improving system energy performance through fault detection and system health monitoring. To lower cost barriers to widespread deployment, a methodology is proposed that takes advantage of existing sensor data, encodes expert knowledge about the application system to create ‘virtual sensors’, and applies statistical and mathematical methods to reduce the time required for manual configurations. The approach combines sensor data points with encoded expert knowledge that is generic to the application system but independent of a particular deployment, thereby reducing the need to tailor to individual deployments. This paper not only presents a method that detects faults from measured energy data, but also (1) describes an engagement method with experts in the energy system domain to identify data, (2) integrates domain knowledge with the data, (3) automatically selects from among the large pool of potential input data, and (4) uses machine learning to automatically build a data-driven fault detection model. Demonstration on a commercial building chiller plant shows that only a small number of virtual sensors is necessary for fault detection with high accuracy rates. This corresponds to the use of only five out of 52 original sensor data points features. With as few as four features, classification F1 scores exceed 90% on the training set and 80% on the testing set. The results are implementable and realizable using off-the-shelf tools. The goal is to design with domain experts an energy monitoring system that can be configured once and then widely deployed with little additional cost or effort
Keywords: Machine learning | Domain knowledge | Time series | Fault detection | Anomaly detection | Energy savings | Energy efficiency
Application of machine learning to accidents detection at directional drilling
کاربرد یادگیری ماشین در تشخیص تصادفات در حفاری جهت دار-2019
We present a data-driven algorithm and mathematical model for anomaly alarming at di- rectional drilling. The algorithm is based on machine learning. It compares the real-time drilling telemetry with one corresponding to past accidents and analyses the level of similar- ity. The model performs a time-series comparison using aggregated statistics and Gradient Boosting classication. It is trained on historical data containing the drilling telemetry of 80 wells drilled within 19 oilelds. The model can detect an anomaly and identify its type by comparing the real-time measurements while drilling with the ones from the database of past accidents. Validation tests show that our algorithm identies half of the anomalies with about 0:53 false alarms per day on average. The model performance ensures sucient time and cost savings as it enables partial prevention of the failures and accidents at the well construction
Keywords: machine learning | anomaly detection | directional drilling | classication | measurements while drilling
Semi-Supervised Learning Based Big Data-Driven Anomaly Detection in Mobile Wireless Networks
تشخیص ناهنجاری های رانده شده با داده های نیمه نظارت بر اساس داده ها در شبکه های بی سیم سیار-2018
With rising capacity demand in mobile networks, the infrastructure is also becoming increasingly denser and complex. This results in collection of larger amount of raw data (big data) that is generated at different levels of network architecture and is typically underutilized. To unleash its full value, innovative machine learning algorithms need to be utilized in order to extract valuable insights which can be used for improving the overall network’s performance. Additionally, a major challenge for network operators is to cope up with increasing number of complete (or partial) cell outages and to simultaneously reduce operational expenditure. This paper contributes towards the aforementioned problems by exploiting big data generated from the core network of 4G LTE-A to detect network’s anomalous behavior. We present a semi-supervised statistical-based anomaly detection technique to identify in time: first, unusually low user activity region depicting sleeping cell, which is a special case of cell outage; and second, unusually high user traffic area corresponding to a situation where special action such as additional resource allocation, fault avoidance solution etc. may be needed. Achieved results demonstrate that the proposed method can be used for timely and reliable anomaly detection in current and future cellular networks.
Keywords: 5G; 4G LTE-A; anomaly detec tion; call detail record; machine learning; big data analytics; network behavior analysis; sleeping cell
The Resilience to Emergencies and Disasters Index: Applying big data to benchmark and validate neighborhood resilience capacity
رفع اشکال در موارد اضطراری و فاجعه: استفاده از داده های بزرگ برای سنجش و اعتبار قابلیت ظرفیت انعطاف پذیری-2018
Resilience planning and emergency management require policymakers and agency leaders to make difficult decisions regarding which at-risk populations should be given priority in the allocation of limited resources. Our work focuses on benchmarking neighborhood resilience by developing a unified, multi-factor index of local and regional resilience capacity: the Resilience to Emergencies and Disasters Index (REDI). The strength of the REDI methodology is the integration of measures of physical, natural, and social systems – operationalized through the collection and analysis of large-scale, heterogeneous, and high resolution urban data – to classify and rank the relative resilience capacity embedded in localized urban systems. Feature selection methodologies are discussed to justify the selection of included indicator variables. Hurricane Sandy is used to validate the REDI scores by measuring the recovery periods for neighborhoods directly impacted by the storm. Using over 12,000,000 re cords for New York City’s 311 service request system, we develop a proxy for neighborhood activity, both pre and post-event. Hurricane Sandy had a significant and immediate impact on neighborhoods classified as least resilient based on the calculated REDI scores, while the most resilient neighborhoods were shown to better withstand disruption to normal activity patterns and more quickly recover to pre-event functional capacity.
Keywords: Urban resilience ، Emergency management ، Disaster recovery ، Community resilience ، Big data ، Anomaly detection
Compression of smart meter big data_ A survey
فشرده سازی داده های بزرگ متریک هوشمند : یک مرور-2018
In recent years, the smart grid has attracted wide attention from around the world. Large scale data are collected by sensors and measurement devices in a smart grid. Smart meters can record fine-grained information about electricity consumption in near real-time, thus forming the smart meter big data. Smart meter big data has provided new opportunities for electric load forecasting, anomaly detection, and demand side management. However, the high-dimensional and massive smart meter big data not only creates great pressure on data transmission lines, but also incur enormous storage costs on data centres. Therefore, to reduce the transmission pressure and storage overhead, improve data mining efficiency, and thus fulfil the potential of smart meter big data. This study presents a comprehensive study on the compression techniques for smart meter big data. The development of smart grids and the characteristics and application challenges of electric power big data are first introduced, followed by analysis of the characteristics and benefits of smart meter big data. Finally, this study focuses on the potential data compression methods for smart meter big data, and discusses the evaluation methods for smart meter big data compression.
Keywords: Smart grid ، Smart meter ، Energy big data ، Data compression
فشرده سازی هوشمند برای داده های بزرگ: مرور
سال انتشار: 2018 - تعداد صفحات فایل pdf انگلیسی: 11 - تعداد صفحات فایل doc فارسی: 40
در سال های اخیر، شبکه هوشمند توجه گسترده ای از سراسر جهان را به خود جلب کرده است. داده های مقیاس بزرگ توسط سنسور ها و دستگاه های اندازه گیری در یک شبکه هوشمند جمع آوری می شوند. مقیاس هوشمند می تواند اطلاعات دقیق در مورد مصرف الکتریسیته را در زمان واقعی به ثبت برساند، بنابراین داده های بزرگ در مقیاس هوشمند اندازه گیری می شود. داده های بزرگ مقیاس هوشمند فرصت های جدیدی برای پیش بینی بار الکتریکی، کشف عادت ها و مدیریت تقاضا ارائه داده است. با این حال، ابعاد بزرگ و داده های بزرگ در مقیاس هوشمند عظیم نه تنها فشار زیادی را بر خطوط انتقال داده ایجاد می کند، بلکه هزینه های ذخیره سازی زیادی را در مراکز داده نیز به همراه می آورد. بنابراین، برای کاهش فشار انتقال و ارتفاع محل ذخیره سازی، برای بهبود راندمان استخراج داده ها، و به اين ترتيب ظرفیت های تحقق هوشمند داده های بزرگ 130 سانتی متری است. مقاله پیش رو یک مطالعه جامع در مورد تکنیک های فشرده سازی داده های بزرگ هوشمند را ارائه می دهد. توسعه شبکه های هوشمند و خصوصیات و چالش های کاربرد داده های بزرگ الکتریکی ابتدا معرفی شده و سپس تجزیه و تحلیل ویژگی ها و مزایای داده های بزرگ مقیاس بزرگ انجام می پذیرد. در نهایت، این مطالعه بر روی روش های فشرده سازی اطلاعات بالقوه برای داده های بزرگ هوشمند تمرکز می کند و روش های ارزیابی فشرده سازی داده های مقیاس هوشمند را مورد بحث قرار می دهد.
کلمات کلیدی: شبکه هوشمند | مقیاس هوشمند | داده های بزرگ انرژی | فشرده سازی داده ها.
|مقاله ترجمه شده|