دانلود مقاله انگلیسی رایگان:برنامه ریزی مقیاس پذیر برای HPC و داده های بزرگ - 2018
اربعین
دانلود مقاله انگلیسی داده های بزرگ رایگان
  • Scalable system scheduling for HPC and big data Scalable system scheduling for HPC and big data
    Scalable system scheduling for HPC and big data

    سال انتشار:

    2018


    عنوان انگلیسی مقاله:

    Scalable system scheduling for HPC and big data


    ترجمه فارسی عنوان مقاله:

    برنامه ریزی مقیاس پذیر برای HPC و داده های بزرگ


    منبع:

    Sciencedirect - Elsevier - J: Parallel Distrib: Comput:, 111 (2018) 76-92: doi:10:1016/j:jpdc:2017:06:009


    نویسنده:

    Albert Reuther *, Chansup Byun, William Arcand, David Bestor, Bill Bergeron, Matthew Hubbell, Michael Jones, Peter Michaleas, Andrew Prout, Antonio Rosa, Jeremy Kepner


    چکیده انگلیسی:

    In the rapidly expanding field of parallel processing, job schedulers are the ‘‘operating systems’’ of modern big data architectures and supercomputing systems. Job schedulers allocate computing resources and control the execution of processes on those resources. Historically, job schedulers were the domain of supercomputers, and job schedulers were designed to run massive, long-running computations over days and weeks. More recently, big data workloads have created a need for a new class of computations consisting of many short computations taking seconds or minutes that process enormous quantities of data. For both supercomputers and big data systems, the efficiency of the job scheduler represents a fundamental limit on the efficiency of the system. Detailed measurement and modeling of the perfor mance of schedulers are critical for maximizing the performance of a large-scale computing system. This paper presents a detailed feature analysis of 15 supercomputing and big data schedulers. For big data workloads, the scheduler latency is the most important performance characteristic of the scheduler. A theoretical model of the latency of these schedulers is developed and used to design experiments targeted at measuring scheduler latency. Detailed benchmarking of four of the most popular schedulers (Slurm, Son of Grid Engine, Mesos, and Hadoop YARN) is conducted. The theoretical model is compared with data and demonstrates that scheduler performance can be characterized by two key parameters: the marginal latency of the scheduler ts and a nonlinear exponent αs. For all four schedulers, the utilization of the computing system decreases to <10% for computations lasting only a few seconds. Multi-level schedulers (such as LLMapReduce) that transparently aggregate short computations can improve utilization for these short computations to >90% for all four of the schedulers that were tested.
    Keywords: Scheduler ، Resource manager ، Job scheduler ، High performance computing ، Data analytics


    سطح: متوسط
    تعداد صفحات فایل pdf انگلیسی: 17
    حجم فایل: 1223 کیلوبایت

    قیمت: رایگان


    توضیحات اضافی:




اگر این مقاله را پسندیدید آن را در شبکه های اجتماعی به اشتراک بگذارید (برای به اشتراک گذاری بر روی ایکن های زیر کلیک کنید)

تعداد نظرات : 0

الزامی
الزامی
الزامی
rss مقالات ترجمه شده rss مقالات انگلیسی rss کتاب های انگلیسی rss مقالات آموزشی