Residential College | false |
Status | 已發表Published |
Multi Resource Scheduling with Task Cloning in Heterogeneous Clusters | |
Xu HL(徐歡樂)1; Yang Liu2; Wing Cheong Lau3 | |
2022-06 | |
Conference Name | International Conference on Parallel Processing |
Source Publication | ICPP 2022 - Proceedings of the 2022 International Conference on Parallel Processing |
Pages | 1-11 |
Conference Date | 2022-8-29 |
Conference Place | Inria |
Country | France |
Publisher | Association for Computing Machinery |
Abstract | To mitigate the straggler effect, today's systems and computing frameworks have adopted redundancy to launch extra copies for stragglers. Two limitations of the existing straggler-mitigation techniques, however, are that resource demand of tasks is only considered in the context of slots and, moreover, redundancy is seldom coordinated with job scheduling. To tackle these issues, in this paper, we present DollyMP, a job scheduler that addresses multi-resource scheduling with task cloning in heterogeneous clusters. DollyMP carefully combines SRPT (Shortest Remaining Processing Time) and SVF (Smallest Volume First) via knapsack optimization to schedule tasks with multi-resource demands and, in the meanwhile, dynamically launches task clones to yield a small job completion time. DollyMP is built on a strong mathematical foundation to guarantee near-optimal performance. The deployment of our Hadoop YARN prototype on a 30-node cluster demonstrates that DollyMP can reduce job response time by 50% under different cluster loads. |
DOI | 10.1145/3545008.3545093 |
Language | 英語English |
Scopus ID | 2-s2.0-85148620520 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Affiliation | 1.University of Macau 2.Shanghai University 3.Chinese University of Hong Kong |
First Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Xu HL,Yang Liu,Wing Cheong Lau. Multi Resource Scheduling with Task Cloning in Heterogeneous Clusters[C]:Association for Computing Machinery, 2022, 1-11. |
APA | Xu HL., Yang Liu., & Wing Cheong Lau (2022). Multi Resource Scheduling with Task Cloning in Heterogeneous Clusters. ICPP 2022 - Proceedings of the 2022 International Conference on Parallel Processing, 1-11. |
Files in This Item: | Download All | |||||
File Name/Size | Publications | Version | Access | License | ||
DollyMP-camera-ready(2072KB) | 会议论文 | 开放获取 | CC BY-NC-SA | View Download |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment