Residential Collegefalse
Status已發表Published
Understanding and Optimizing Workloads for Unified Resource Management in Large Cloud Platforms
Lu, Chengzhi1; Xu, Huanle2; Ye, Kejiang3; Xu, Guoyao4; Zhang, Liping4; Yang, Guodong4; Xu, Chengzhong2
2023-05-08
Conference Name18th European Conference on Computer Systems, EuroSys 2023
Source PublicationProceedings of the 18th European Conference on Computer Systems, EuroSys 2023
Pages416-432
Conference Date2023/05/08-2023/05/12
Conference PlaceRome
PublisherAssociation for Computing Machinery, Inc
Abstract

To fully utilize computing resources, cloud providers such as Google and Alibaba choose to co-locate online services with batch processing applications in their data centers. By implementing unified resource management policies, different types of complex computing jobs request resources in a consistent way, which can help data centers achieve global optimal scheduling and provide computing power with higher quality. To understand this new scheduling paradigm, in this paper, we first present an in-depth study of Alibaba’s unified scheduling workloads. Our study focuses on the characterization of resource utilization, the application running performance, and scheduling scalability. We observe that although computing resources are significantly over-committed under unified scheduling, the resource utilization in Alibaba data centers is still low. In addition, existing resource usage predictors tend to make severe overestimations. At the same time, tasks within the same application behave fairly consistently, and the running performance of tasks can be well-profiled with respect to resource contention on the corresponding physical host. Based on these observations, in this paper, we design Optum, a unified data center scheduler for improving the overall resource utilization while ensuring good performance for each application. Optum formulates an optimization problem to schedule unified task requests, aiming to balance the trade-off between utilization and resource contention. Optum also implements efficient heuristics to solve the optimization problem in a scalable manner. Large-scale experiments demonstrate that Optum can save up to 15% of resources without performance degradation compared to state-of-the-art unified scheduling schemes.

KeywordCloud Computing Resource Over-commitment Unified Scheduling
DOI10.1145/3552326.3587437
URLView the original
Indexed ByCPCI-S
Funding ProjectSoftware-defined Methods and Key Technologies for Intelligent Control of Cloud Data Centres ; Efficient Integration and Dynamic Cognitive Technology and Platform for Urban Public Services
Language英語English
WOS Research AreaComputer Science
WOS SubjectComputer Science, Hardware & Architecture ; Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods
WOS IDWOS:001062106700026
Scopus ID2-s2.0-85160209377
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Faculty of Science and Technology
Affiliation1.Shenzhen Institute of Advanced Technology, CAS Univ. of CAS, Univ. of Macau Macau SAR, China
2.University of Macau Macau SAR, China
3.Shenzhen Institute of Advanced Technology, CAS, Shenzhen, China
4.Alibaba Group, Hangzhou, China
First Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Lu, Chengzhi,Xu, Huanle,Ye, Kejiang,et al. Understanding and Optimizing Workloads for Unified Resource Management in Large Cloud Platforms[C]:Association for Computing Machinery, Inc, 2023, 416-432.
APA Lu, Chengzhi., Xu, Huanle., Ye, Kejiang., Xu, Guoyao., Zhang, Liping., Yang, Guodong., & Xu, Chengzhong (2023). Understanding and Optimizing Workloads for Unified Resource Management in Large Cloud Platforms. Proceedings of the 18th European Conference on Computer Systems, EuroSys 2023, 416-432.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Lu, Chengzhi]'s Articles
[Xu, Huanle]'s Articles
[Ye, Kejiang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Lu, Chengzhi]'s Articles
[Xu, Huanle]'s Articles
[Ye, Kejiang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Lu, Chengzhi]'s Articles
[Xu, Huanle]'s Articles
[Ye, Kejiang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.