AN EFFECTIVE JOB SCHEDULING ALGORITHM IN HIGH PERFORMANCE COMPUTING

Abstract

The need for efficient resource management in distributed computing systems is greater than ever due to the significant growth of High Performance Computing (HPC) and its predicted future expansion. The increasing volume of computing workloads requires the application of effective resource management and workload scheduling in order to maximize resource utilization while maintaining a reasonable level of computational performance. It is difficult to effectively schedule workloads on resources while maintaining performance requirements. Furthermore, resource management becomes much more difficult in real-world situations due to non- clairvoyance regarding task dimensions. Using the latest machine learning and data science tools, the suggested study methodology explores the scheduling problem that is compatible for HPC and explores the difficulties in implementing the scheduling in real-world scenarios. The majority of recent scientific and engineering advancements are attributed to high performance computing. High performance systems, which are massive parallel networked supercomputers, are necessary for research conducted at national laboratories. In a world where data is becoming more and more important, High Performance Computing (HPC) systems are becoming essential tools for solving complex, compute-intensive, and data-intensive problems in a variety of engineering, business, and scientific domains. This has led to new discoveries in science and technology as well as the development of more dependable and effective goods and services. In High Performance Computing (HPC) platforms, job scheduling is a challenging problem with unknowns regarding the arrival process and execution time of jobs. This proposed system is try to improve Min-min and priority algorithm based on combination of existing two algorithms. Various methods are searching for scheduling using High Performance Computing (HPC) in research trend. The proposed system will show the comparison among Min-min, priority and hybrid approach for job scheduling system. The performance of different HPC systems are evaluated by Quality of Service (QoS) metrics (turnaround time, throughput and response time). To test the effectiveness and correctness of simulator, experiments will run to gather statistics. The workload logs used in the experiments are the HPC2N log, KIT-FH2-2016-1log, and the SDSC-SP2-1998-4.2-cln log given in the Parallel Workloads Archive.