Consistent Hashing with Bounded Loads
Skip to main content
Explore our many areas of focus
Explore all research areas
Applied AI & sciences
Earth AI
Health AI
Science AI
Sustainability & crisis resilience
Foundational ML & algorithms
Algorithms & theory
Information retrieval
Machine intelligence
Machine perception
Natural language processing
People, systems & quantum AI
Human-computer interaction and visualization
Networking
Quantum AI
Responsible AI
Anti abuse
Software engineering
Software systems
Learn More
Publications
Projects
Building a collaborative ecosystem
Datasets
Access high-quality datasets to accelerate your research.
Tools & services
Explore our latest AI models and products.
Open source
Discover open-source code and collaborate with the community.
Shaping the future together
See all programs
Faculty programs
Participating in the academic research community through meaningful engagement with university faculty.
Student programs
Supporting the next generation of researchers through a wide range of programming.
Locations
Find your place in our global offices and research labs.
Translating discovery into real-world impact
People
Our researchers drive advancements in computer science through both fundamental and applied research.
Teams
Collaborative groups tackling the world's most challenging AI problems.
Research
Explore our many areas of focus
Explore all research areas
Applied AI & sciences
Earth AI
Health AI
Science AI
Sustainability & crisis resilience
Foundational ML & algorithms
Algorithms & theory
Information retrieval
Machine intelligence
Machine perception
Natural language processing
People, systems & quantum AI
Human-computer interaction and visualization
Networking
Quantum AI
Responsible AI
Anti abuse
Software engineering
Software systems
Learn More
Publications
Projects
Resources
Building a collaborative ecosystem
Datasets
Access high-quality datasets to accelerate your research.
Tools & services
Explore our latest AI models and products.
Open source
Discover open-source code and collaborate with the community.
Conferences & events
Careers
Shaping the future together
See all programs
Faculty programs
Participating in the academic research community through meaningful engagement with university faculty.
Student programs
Supporting the next generation of researchers through a wide range of programming.
Locations
Find your place in our global offices and research labs.
Blog
About
Translating discovery into real-world impact
People
Our researchers drive advancements in computer science through both fundamental and applied research.
Teams
Collaborative groups tackling the world's most challenging AI problems.
Google
Research
Google AI
Learn about all our AI
Google DeepMind
Explore the frontier of AI
Google Labs
Try our AI experiments
Research
Resources
Conferences & events
Careers
Blog
About
Search
Home
Publications
Consistent Hashing with Bounded Loads
Vahab Mirrokni
Mikkel Thorup
Morteza Zadimoghaddam
Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms (2018), pp. 587-604
Download
Google Scholar
Copy Bibtex
Abstract
Designing algorithms for balanced allocation of clients to servers in dynamic settings is a challenging problem for a variety of reasons. Both servers and clients may be added and/or removed from the system periodically, and the main objectives of allocation algorithms are: the uniformity of the allocation, and the number of moves after adding or removing a server or a client. The most popular solution for our dynamic settings is Consistent Hashing. However, the load balancing of consistent hashing is no better than a random assignment of clients to servers, so with n of each, we expect many servers to be overloaded with Θ(logn/loglogn) clients. In this paper, with n clients and n servers, we get a guaranteed max-load of 2 while only moving an expected constant number of clients for each update.
We take an arbitrary user specified balancing parameter c=1+ϵ>1. With m balls and n bins in the system, we want no load above ⌈cm/n⌉. Meanwhile we want to bound the expected number of balls that have to be moved when a ball or server is added or removed. Compared with general lower bounds without capacity constraints, we show that in our algorithm when a ball or bin is inserted or deleted, the expected number of balls that have to be moved is increased only by a multiplicative factor O(1ϵ2) for ϵ≤1 (Theorem 4) and by a factor 1+O(logcc) for ϵ≥1 (Theorem 3). Technically, the latter bound is the most challenging to prove. It implies that we for superconstant c only pay a negligible cost in extra moves. We also get the same bounds for the simpler problem where we instead of a user specified balancing parameter have a fixed bin capacity C for all bins.
Research Areas
Algorithms and theory
Meet the teams driving innovation
Our teams advance the state of the art through research, systems engineering, and collaboration across Google.
See our teams
Follow us
Explore our other initiatives
Google AI
Discover how Google AI is committed to enriching knowledge and solving complex challenges
Products
Build
Research
Responsibility
Societal Impact
About
Google Cloud
High-performance infrastructure for cloud computing, data analytics & machine learning
Overview
Solutions
Products
Pricing
Resources
Google DeepMind
Our mission is to build AI responsibly to benefit humanity
Models
Research
Science
About
Google Labs
Explore the future of AI responsibly with Google Labs
About
Experiments
Stay connected
About Google
Google Products
Privacy
Terms
Cookies management controls
×