Data Mining Strategies for Real-Time Control in New York City
استراتژی داده کاوی برای کنترل زمان واقعی در شهر نیویورک-2105
The Data Mining System (DMS) at New York City Department of Transportation (NYCDOT) mainly consists of four database systems for traffic and pedestrian/bicycle volumes, crash data, and signal timing plans as well as the Midtown in Motion (MIM) systems which are used as part of the NYCDOT Intelligent Transportation System (ITS) infrastructure. These database and control systems are operated by different units at NYCDOT as an independent database or operation system. New York City experiences heavy traffic volumes, pedestrians and cyclists in each Central Business District (CBD) area and along key arterial systems. There are consistent and urgent needs in New York City for real-time control to improve mobility and safety for all users of the street networks, and to provide a timely response and management of random incidents. Therefore, it is necessary to develop an integrated DMS for effective real-time control and active transportation management (ATM) in New York City. This paper will present new strategies for New York City suggesting the development of efficient and cost-effective DMS, involving: 1) use of new technology applications such as tablets and smartphone with Global Positioning System (GPS) and wireless communication features for data collection and reduction; 2) interface development among existing database and control systems; and 3) integrated DMS deployment with macroscopic and mesoscopic simulation models in Manhattan. This study paper also suggests a complete data mining process for real-time control with traditional static data, current real timing data from loop detectors, microwave sensors, and video cameras, and new real-time data using the GPS data. GPS data, including using taxi and bus GPS information, and smartphone applications can be obtained in all weather conditions and during anytime of the day. GPS data and smartphone application in NYCDOT DMS is discussed herein as a new concept. © 2014 The Authors. Published by Elsevier B.V. Selection and peer-review under responsibility of Elhadi M. Shakshu Keywords: Data Mining System (DMS), New York City, real-time control, active transportation management (ATM), GPS data
Potentials, trends, and prospects in edge technologies: Fog, cloudlet, mobile edge, and micro data centers
پتانسیل ها، گرایشات و چشم اندازها در روشهای لبه ای: مراکز داده ای مات، تکه ابر، لبه ای سیار و میکرو-2018
Advancements in smart devices, wearable gadgets, sensors, and communication paradigm have enabled the vision of smart cities, pervasive healthcare, augmented reality and interactive multimedia, Internet of Every Thing (IoE), and cognitive assistance, to name a few. All of these visions have one thing in common, i.e., delay sensitivity and instant response. Various new technologies designed to work at the edge of the network, such as fog computing, cloudlets, mobile edge computing, and micro data centers have emerged in the near past. We use the name ``edge computing for this set of emerging technologies. Edge computing is a promising paradigm to offer the required computation and storage resources with minimal delays because of ``being near to the users or terminal devices. Edge computing aims to bring cloud resources and services at the edge of the network, as a middle layer between end user and cloud data centers, to offer prompt service response with minimal delay. Two major aims of edge computing can be denoted as: (a) minimize response delay by servicing the users’ request at the network edge instead of servicing it at far located cloud data centers, and (b) minimize downward and upward traffic volumes in the network core. Minimization of network core traffic inherently brings energy efficiency and data cost reductions. Downward network traffic can be minimized by servicing set of users at network edge instead of service providers data centers (e.g., multimedia and shared data) Content Delivery Networks (CDNs), and upward traffic can be minimized by processing and filtering raw data (e.g., sensors monitored data) and uploading the processed information to cloud. This survey presents a detailed overview of potentials, trends, and challenges of edge computing. The survey illustrates a list of most significant applications and potentials in the area of edge computing. State of the art literature on edge computing domain is included in the survey to guide readers towards the current trends and future opportunities in the area of edge computing.
keywords: Edge computing| Fog computing| Internet of Things
QoE-Driven Big Data Management in Pervasive Edge Computing Environment
مدیریت داده های بزرگ بر مبنای QoE در محدوده محاسباتی فراگیر لبه-2018
In the age of big data, services in the pervasive edge environment are expected to offer end-users better Quality-of-Experience (QoE) than that in a normal edge environment. However, the combined impact of the storage, delivery, and sensors used in various types of edge devices in this environment is producing volumes of high-dimensional big data that are increasingly pervasive and redundant. Therefore, enhancing the QoE has become a major challenge in high-dimensional big data in the pervasive edge computing environment. In this paper, to achieve high QoE, we propose a QoE model for evaluating the qualities of services in the pervasive edge computing environment. The QoE is related to the accuracy of high-dimensional big data and the transmission rate of this accurate data. To realize high accuracy of high-dimensional big data and the transmission of accurate data through out the pervasive edge computing environment, in this study we focused on the following two aspects. First, we formulate the issue as a high-dimensional big data management problem and test different transmission rates to acquire the best QoE. Then, with respect to accuracy, we propose a Tensor-Fast Convolutional Neural Network (TF-CNN) algorithm based on deep learning, which is suitable for high-dimensional big data analysis in the pervasive edge computing environment. Our simulation results reveal that our proposed algorithm can achieve high QoE performance.
Key words: Quality-of-Experience (QoE); high-dimensional big data management; deep learning; pervasive edge computing
Big Data Architectures and the Internet of Things: A Systematic Mapping Study
معماری های داده های بزرگ و اینترنت اشیا: یک مطالعه نقشه ای سیستماتیک-2018
Big Data and IoT have made huge strides in detection technologies, resulting in "smart" devices consisting of sensors, and massive data processing. So far, there is no common strategy for designing Big Data architectures containing IoT, since they depend on the context of the problem to be solved. But in recent years, various architectures have been proposed that serve as examples for future research in this area. The aim of this article is to provide an overview of the architectures published so far, serving as a starting point for future research. The methodology used is that of systematic mapping
Keywords : Big Data; architecture; Internet of Things; IoT; systematic mapping
Conception and Exploration of Using Data as a Service in Tunnel Construction with the NATM
مفهوم و اکتشاف استفاده از داده ها به عنوان یک سرویس در ساخت تونل با NATM-2018
The New Austrian Tunneling Method (NATM) has been widely used in the construction of mountain tun nels, urban metro lines, underground storage tanks, underground power houses, mining roadways, and so on. The variation patterns of advance geological prediction data, stress–strain data of supporting struc tures, and deformation data of the surrounding rock are vitally important in assessing the rationality and reliability of construction schemes, and provide essential information to ensure the safety and scheduling of tunnel construction. However, as the quantity of these data increases significantly, the uncertainty and discreteness of the mass data make it extremely difficult to produce a reasonable con struction scheme; they also reduce the forecast accuracy of accidents and dangerous situations, creating huge challenges in tunnel construction safety. In order to solve this problem, a novel data service system is proposed that uses data-association technology and the NATM, with the support of a big data environ ment. This system can integrate data resources from distributed monitoring sensors during the construc tion process, and then identify associations and build relations among data resources under the same construction conditions. These data associations and relations are then stored in a data pool. With the development and supplementation of the data pool, similar relations can then be used under similar con ditions, in order to provide data references for construction schematic designs and resource allocation. The proposed data service system also provides valuable guidance for the construction of similar projects.
Keywords: New Austrian Tunneling Method ، Big data environments ، Data as a service ، Tunnel construction
Big Data Model Simulation on a Graph Database for Surveillance in Wireless Multimedia Sensor Networks
شبیه سازی مدل داده های بزرگ بر روی یک پایگاه داده گراف برای نظارت بر شبکه های حسگر چندرسانه ای بیسیم-2018
Sensors are present in various forms all around the world such as mobile phones, surveillance cameras, smart televisions, intelligent refrigerators and blood pressure monitors. Usually, most of the sensors are a part of some other system with similar sensors that compose a network. One of such networks is composed of millions of sensors connected to the Internet which is called Internet of Things (IoT). With the advances in wireless communication technologies, multimedia sensors and their networks are expected to be major components in IoT. Many studies have already been done on wireless multimedia sensor networks in diverse domains like fire detection, city surveillance, early warning systems, etc. All those applications position sensor nodes and collect their data for a long time period with real-time data flow, which is considered as big data. Big data may be structured or unstructured and needs to be stored for further processing and analyzing. Analyzing multimedia big data is a challenging task requiring a high-level modeling to efficiently extract valuable information/knowledge from data. In this study, we propose a big database model based on graph database model for handling data generated by wireless multimedia sensor networks. We introduce a simulator to generate synthetic data and store and query big data using graph model as a big database. For this purpose, we evaluate the well-known graph-based NoSQL databases, Neo4j and OrientDB, and a relational database, MySQL. We have run a number of query experiments on our implemented simulator to show that which database system(s) for surveillance in wireless multimedia sensor networks is efficient and scalable.
Keywords: Internet of things (IoT) ، Big graph databases ، NoSQL databases ، Wireless multimedia sensor networks ، Simulator
A dynamic neural network architecture with immunology inspired optimization for weather data forecasting
یک معماری شبکه عصبی پویا با ایمنولوژی بهینه سازی برای پیش بینی داده های آب و هوایی-2018
Recurrent neural networks are dynamical systems that provide for memory capabilities to recall past behaviour, which is necessary in the prediction of time series. In this paper, a novel neural network architecture inspired by the immune algorithm is presented and used in the forecasting of naturally occurring signals, including weather big data signals. Big Data Analysis is a major research frontier, which attracts extensive attention from academia, industry and government, particularly in the context of handling issues related to complex dynamics due to changing weather conditions. Recently, extensive deployment of IoT, sensors, and ambient intelligence systems led to an exponential growth of data in the climate domain. In this study, we concentrate on the analysis of big weather data by using the Dynamic Self Organized Neural Network Inspired by the Immune Algorithm. The learning strategy of the network focuses on the local properties of the signal using a self-organised hidden layer inspired by the immune algorithm, while the recurrent links of the network aim at recalling previously observed signal patterns. The proposed network exhibits improved performance when compared to the feedforward multilayer neural network and state-of-the-art recurrent networks, e.g., the Elman and the Jordan networks. Three non-linear and non-stationary weather signals are used in our experiments. Firstly, the signals are transformed into stationary, followed by 5-steps ahead prediction. Improvements in the prediction results are observed with respect to the mean value of the error (RMS) and the signal to noise ratio (SNR), however to the expense of additional computational complexity, due to presence of recurrent links.
Keywords: Recurrent Neural Networks ،Immune Systems Optimisation، Time Series Data analytics ، weather forecasting
Improving early OSV design robustness by applying Multivariate Big Data Analytics on a ships life cycle
بهبود استحکام طراحی اولیه OSV با استفاده از «تجزیه و تحلیل داده های چند متغیره» در یک چرخه عمر کشتی-2018
Typically, only a smaller portion of the monitorable operational data (e.g. from sensors and environment) from Offshore Support Vessels (OSVs) are used at present. Operational data, in addition to equipment performance data, design and construction data, creates large volumes of data with high veracity and variety. In most cases, such data richness is not well understood as to how to utilize it better during design and operation. It is, very often, too time consuming and resource demanding to estimate the final operational performance of vessel concept design solution in early design by applying simulations and model tests. This paper argues that there is a significant potential to integrate ship lifecycle data from different phases of its operation in large data repository for deliberate aims and evaluations. It is disputed discretely in the paper, evaluating performance of real similar type vessels during early stages of the design process, helps substantially improving and fine-tuning the per formance criterion of the next generations of vessel design solutions. Producing learning from such a ship lifecycle data repository to find useful patterns and relationships among design parameters and existing fleet real performance data, requires the implementation of modern data mining techniques, such as big data and clus tering concepts, which are introduced and applied in this paper. The analytics model introduced suggests and reviews all relevant steps of data knowledge discovery, including pre-processing (integration, feature selection and cleaning), processing (data analyzing) and post processing (evaluating and validating results) in this context.
Keywords: External data ، Internal data ، Abnormality ، Missing data ، Outliers ، Randomness ، Multivariate analysis ، Data integration ، Clustering
Compression of smart meter big data_ A survey
فشرده سازی داده های بزرگ متریک هوشمند : یک مرور-2018
In recent years, the smart grid has attracted wide attention from around the world. Large scale data are collected by sensors and measurement devices in a smart grid. Smart meters can record fine-grained information about electricity consumption in near real-time, thus forming the smart meter big data. Smart meter big data has provided new opportunities for electric load forecasting, anomaly detection, and demand side management. However, the high-dimensional and massive smart meter big data not only creates great pressure on data transmission lines, but also incur enormous storage costs on data centres. Therefore, to reduce the transmission pressure and storage overhead, improve data mining efficiency, and thus fulfil the potential of smart meter big data. This study presents a comprehensive study on the compression techniques for smart meter big data. The development of smart grids and the characteristics and application challenges of electric power big data are first introduced, followed by analysis of the characteristics and benefits of smart meter big data. Finally, this study focuses on the potential data compression methods for smart meter big data, and discusses the evaluation methods for smart meter big data compression.
Keywords: Smart grid ، Smart meter ، Energy big data ، Data compression
فشرده سازی هوشمند برای داده های بزرگ: مرور
سال انتشار: 2018 - تعداد صفحات فایل pdf انگلیسی: 11 - تعداد صفحات فایل doc فارسی: 40
در سال های اخیر، شبکه هوشمند توجه گسترده ای از سراسر جهان را به خود جلب کرده است. داده های مقیاس بزرگ توسط سنسور ها و دستگاه های اندازه گیری در یک شبکه هوشمند جمع آوری می شوند. مقیاس هوشمند می تواند اطلاعات دقیق در مورد مصرف الکتریسیته را در زمان واقعی به ثبت برساند، بنابراین داده های بزرگ در مقیاس هوشمند اندازه گیری می شود. داده های بزرگ مقیاس هوشمند فرصت های جدیدی برای پیش بینی بار الکتریکی، کشف عادت ها و مدیریت تقاضا ارائه داده است. با این حال، ابعاد بزرگ و داده های بزرگ در مقیاس هوشمند عظیم نه تنها فشار زیادی را بر خطوط انتقال داده ایجاد می کند، بلکه هزینه های ذخیره سازی زیادی را در مراکز داده نیز به همراه می آورد. بنابراین، برای کاهش فشار انتقال و ارتفاع محل ذخیره سازی، برای بهبود راندمان استخراج داده ها، و به اين ترتيب ظرفیت های تحقق هوشمند داده های بزرگ 130 سانتی متری است. مقاله پیش رو یک مطالعه جامع در مورد تکنیک های فشرده سازی داده های بزرگ هوشمند را ارائه می دهد. توسعه شبکه های هوشمند و خصوصیات و چالش های کاربرد داده های بزرگ الکتریکی ابتدا معرفی شده و سپس تجزیه و تحلیل ویژگی ها و مزایای داده های بزرگ مقیاس بزرگ انجام می پذیرد. در نهایت، این مطالعه بر روی روش های فشرده سازی اطلاعات بالقوه برای داده های بزرگ هوشمند تمرکز می کند و روش های ارزیابی فشرده سازی داده های مقیاس هوشمند را مورد بحث قرار می دهد.
کلمات کلیدی: شبکه هوشمند | مقیاس هوشمند | داده های بزرگ انرژی | فشرده سازی داده ها.
|مقاله ترجمه شده|