با سلام خدمت کاربران در صورتی که با خطای سیستم پرداخت بانکی مواجه شدید از طریق کارت به کارت (6037997535328901 بانک ملی ناصر خنجری ) مقاله خود را دریافت کنید (تا مشکل رفع گردد).
دسته بندی:
داده های بزرگ - big data
سال انتشار:
2016
عنوان انگلیسی مقاله:
On Traffic-Aware Partition and Aggregation in MapReduce for Big Data Applications
ترجمه فارسی عنوان مقاله:
تجمیع و تقسیم بندی ترافیک آگاه در MapReduce برای کاربردهای داده های بزرگ
منبع:
IEEE - IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL: 27, NO: 3, MARCH 2016
نویسنده:
Huan Ke, Peng Li, Song Guo,Minyi Guo
چکیده انگلیسی:
The MapReduce programming model simplifies large-scale data processing on commodity cluster by exploiting parallel
map tasks and reduce tasks. Although many efforts have been made to improve the performance of MapReduce jobs, they ignore the
network traffic generated in the shuffle phase, which plays a critical role in performance enhancement. Traditionally, a hash function is
used to partition intermediate data among reduce tasks, which, however, is not traffic-efficient because network topology and data size
associated with each key are not taken into consideration. In this paper, we study to reduce network traffic cost for a MapReduce job by
designing a novel intermediate data partition scheme. Furthermore, we jointly consider the aggregator placement problem, where each
aggregator can reduce merged traffic from multiple map tasks. A decomposition-based distributed algorithm is proposed to deal with
the large-scale optimization problem for big data application and an online algorithm is also designed to adjust data partition and
aggregation in a dynamic manner. Finally, extensive simulation results demonstrate that our proposals can significantly reduce network
traffic cost under both offline and online cases.
Index Terms: MapReduce | partition | aggregation | big data | lagrangian decomposition
قیمت: رایگان
توضیحات اضافی:
تعداد نظرات : 0