List of Contents
- The Background
- What is a Rate Limiter?
- Why is it Needed?
- How to Implement Rate Limiting Logic
- The Top 5 Rate Limiting Algorithms
The Background
Before learning about rate limiting, let us understand where systems use it and why they need it.
In a typical system, a frontend talks to a backend server. When users send requests, problems can occur. Especially when they send many requests within a short period of time. These issues may include excessive API requests, uneven spacing between requests, sudden spikes in traffic (bursts), and increased load on backend components such as databases.
While these problems can indirectly affect performance, the primary purpose of rate limiting is not performance optimization. Instead, rate limiting is a logic or piece of code that protects a system from abuse, traffic spikes, overload, and unfair usage, ensuring that all users get fair access to system resources.
What is a Rate Limiter?
Rate limiting is a mechanism that controls how many requests a client can make to a server within a specific time window. For example, you might allow 10 requests per minute.
It does not directly make the system faster. Instead, it protects the system from overload, abuse, and unfair usage by regulating incoming traffic.
Some of the problems include the following
- Hitting an API frequently by a large number of users (count/frequency).
- Sudden spikes in requests (Bursts).
- Spacing or relaxation between each request is very little (smoothness).
It acts like a security guard at an airport. The guard checks every passenger before they reach the gate. In technical terms, we call this a middleware. It evaluates the request before it reaches your core services or controllers.
Why Is It Needed?
As discussed above, a rate limiter does not directly improve system performance. Instead, it helps maintain system stability, fairness, and reliability, which indirectly leads to a better user experience.
When a large number of users are requesting services from a system, it makes the system consume more computational resources. Therefore, this increases the infrastructure (servers) cost and risk of overload.
Instead of adding more servers, a better option is to control incoming traffic with a rate limiter. When we implement rate limiting logic as middleware, it evaluates each request before the request reaches controllers or services, protecting the system from overuse and abusive traffic.
How to Implement Rate Limiting Logic
Now we have come to the understanding of why rate limiting is used and how it helps ensure efficient resource utilization. We now go to the next step on how to implement, and what are the main important factors to keep in mind before implementing this as middleware.
Things to keep in mind before implementing the logic,
- Does the system only require basic control on the frequency of requests?
- Should it allow or restrict sudden bursts of requests?
- Does the system need smoothness in request handling?
- Does the system need to control how fast an individual user can send requests?
After noting down all the requirements, the next step is to think about how to implement a rate limiter as a middleware. We have many algorithms to implement the logic which provides their own advantages with each algorithm. The various algorithms and their pros and cons will be discussed in the next section.
The Top 5 Rate Limiting Algorithms
The algorithms that can be used to implement the rate limiter are:
- Fixed Window Counter Algorithm
- Leaky Bucket Algorithm
- Token Bucket Algorithm
- Sliding Window Log Algorithm
- Sliding Window Counter Algorithm
Each algorithm handles requests, memory use, bursts, and traffic flow differently. Therefore, the choice of algorithm depends on business requirements, traffic patterns, and system scale.
Overall, these algorithms help regulate request frequency, control traffic spikes, and ensure fair usage. As a result, they play a key role in maintaining system stability and efficient resource utilization.
To learn more about the implementations of these algorithms, follow this link
Leave a Reply