Going Fast and Fair: Latency Optimization for Cloud-Based Service Chains

Yuchao Zhang, Ke Xu, Haiyang Wang, Qi Li, Tong Li, Xuan Cao

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


State-of-the-art microservices have been attracting more attention in recent years. A broad spectrum of online interactive applications are now programmed to service chains on the cloud, seeking better system scalability and lower operating costs. Different from the conventional batch jobs, most of these applications consist of multiple stand-alone services that communicate with each other. These step-by-step operations unavoidably introduce higher latency to the delay-sensitive chained services. In this article, we aim at designing an optimization approach for reducing the latency of chained services. Specifically, presenting the measurement and analysis of chained services on Baidu's cloud platform, our real-world trace indicates that these chained services are suffering from significantly high latency because they are mostly handled by different queues on cloud servers for multiple times. However, such a unique feature introduces significant challenges to optimize a microservice's overall queueing delay. To address this problem, we propose a delay-guaranteed approach to accelerate the overall queueing of chained services while obtaining fairness across all the workloads. Our evaluations on Baidu servers shows that the proposed design can successfully reduce the latency of chained services by 35 percent with minimal impact on other workloads.

Original languageEnglish (US)
Pages (from-to)138-143
Number of pages6
JournalIEEE Network
Issue number2
StatePublished - Mar 1 2018

Bibliographical note

Funding Information:
AcknoWledgMent This work was supported by the National Natural Foundation of China (61472212 and 61572278), EU Marie Curie Actions CROWN (FP7-PEOPLE-2013-IRSES-610524), the National Key R&D Program of China (2016YFB0800102), and the R&D Program of Shenzhen (JCYJ20170307153259323).

Publisher Copyright:
© 2018 IEEE.


Dive into the research topics of 'Going Fast and Fair: Latency Optimization for Cloud-Based Service Chains'. Together they form a unique fingerprint.

Cite this