با سلام خدمت کاربران در صورتی که با خطای سیستم پرداخت بانکی مواجه شدید از طریق کارت به کارت (6037997535328901 بانک ملی ناصر خنجری ) مقاله خود را دریافت کنید (تا مشکل رفع گردد).
ردیف | عنوان | نوع |
---|---|---|
1 |
Learning-Based Memory Allocation Optimization for Delay-Sensitive Big Data Processing
بهینه سازی اختصاص حافظه بر مبنای یادگیری برای پردازش داده های بزرگ با حساسیت تاخیری-2018 Optimal resource provisioning is essential for scalable big data analytics. However, it has been difficult to accurately
forecast the resource requirements before the actual deployment of these applications as their resource requirements are heavily
application and data dependent. This paper identifies the existence of effective memory resource requirements for most of the big data
analytic applications running inside JVMs in distributed Spark environments. Provisioning memory less than the effective memory
requirement may result in rapid deterioration of the application execution in terms of its total execution time. A machine learning-based
prediction model is proposed in this paper to forecast the effective memory requirement of an application given its service level
agreement. This model captures the memory consumption behavior of big data applications and the dynamics of memory utilization in
a distributed cluster environment. With an accurate prediction of the effective memory requirement, it is shown that up to 60 percent
savings of the memory resource is feasible if an execution time penalty of 10 percent is acceptable. The accuracy of the model is
evaluated on a physical Spark cluster with 128 cores and 1TB of total memory. The experiment results show that the proposed solution
can predict the minimum required memory size for given acceptable delays with high accuracy, even if the behavior of target
applications is unknown during the training of the model.
Index Terms: Big data, spark, memory over-commitment, garbage collection, profiling, modeling, performance-cost tradeoff |
مقاله انگلیسی |