How-To: Rack Middleware for API Throttling
I will show you a technique to impose a rate limit (aka API Throttling) on a Ruby Web Service. I will be using Rack middleware so you can use this no matter what Ruby Web Framework you are using, as long as it is Rack-compliant.
Introduction to Rack
There are plenty of great resources to learn the basic of Rack so I will not be explaining how Rack works here but you will need to understand it in order to follow this post. I highly recommend watching the three Rack screencasts from Remi to get started with Rack.
Basic Rack Application
First, make sure you have the thin webserver installed.
sudo gem install thin
We are going to use the following ‘Hello World’ Rack application to test our API Throttling middleware.
use Rack::ShowExceptions
use Rack::Lint
run lambda {|env| [200, { 'Content-Type' => 'text/plain', 'Content-Length' => '12' }, ["Hello World!"] ] }
Save this code in a file called config.ru and then you can run it with the thin webserver, using the following command:
thin --rackup config.ru start
Now you can open another terminal window (or a browser) to test that this is working as expected:
curl -i http://localhost:3000
The -i option tells curl to include the HTTP-header in the output so you should see the following:
$ curl -i http://localhost:3000 HTTP/1.1 200 OK Content-Type: text/plain Content-Length: 12 Connection: keep-alive Server: thin 1.0.0 codename That's What She Said Hello World!
At this point, we have a basic rack application that we can use to test our rack middleware. Now let’s get started.
Redis
We need a way to memorize the number of requests that users are making to our web service if we want to limit the rate at which they can use the API. Every time they make a request, we want to check if they’ve gone past their rate limit before we respond to the request. We also want to store the fact that they’ve just made a request. Since every call to our web service requires this check and memorization process, we would like this to be done as fast as possible.
This is where Redis comes in. Redis is a super-fast key-value database that we’ve highlighted in a previous blog post. It can do about 110,000 SETs per second, about 81,000 GETs per second. That’s the kind of performance that we are looking for since we would not like our ‘rate limiting’ middleware to reduce the performance of our web service.
Our Rack Middleware
We are assuming that the web service is using HTTP Basic Authentication. You could use another type of authentication and adapt the code to fit your model.
Our rack middleware will do the following:
- For every request received, increment a key in our database. The key string will consists of the authenticated username followed by a timestamp for the current hour. For example, for a user called joe, the key would be: joe_2009-05-01-12
- If the value of that key is less than our ‘maximum requests per hour limit’, then return an HTTP Response with a status code of 503, indicating that the user has gone over his rate limit.
- If the value of the key is less than the maximum requests per hour limit, then allow the user’s request to go through.
Redis has an atomic INCR command that is the perfect fit for our use case. It increments the key value by one. If the key does not exist, it sets the key to the value of “0″ and then increments it. Awesome! We don’t even need to write our own logic to check if the key exists before incrementing it, Redis takes care of that for us.
r = Redis.new
key = "#{auth.username}_#{Time.now.strftime("%Y-%m-%d-%H")}"
r.incr(key)
return over_rate_limit if r[key].to_i > @options[:requests_per_hour]
If our redis-server is not running, rather than throwing an error affecting all our users, we will let all the requests pass through by catching the exception and doing nothing. That means that if your redis-server goes down, you are no longer throttling the use of your web service so you need to make sure it’s always running (using monit or god, for example).
Finally, we want anyone who might use this Rack middleware to be able to set their limit via the requests_per_hour option.
The full code for our middleware is below. You can also find it at github.com/dambalah/api-throttling.
require 'rubygems'
require 'rack'
require 'redis'
class ApiThrottling
def initialize(app, options={})
@app = app
@options = {:requests_per_hour => 60}.merge(options)
end
def call(env, options={})
auth = Rack::Auth::Basic::Request.new(env)
if auth.provided?
return bad_request unless auth.basic?
begin
r = Redis.new
key = "#{auth.username}_#{Time.now.strftime("%Y-%m-%d-%H")}"
r.incr(key)
return over_rate_limit if r[key].to_i > @options[:requests_per_hour]
rescue Errno::ECONNREFUSED
# If Redis-server is not running, instead of throwing an error, we simply do not throttle the API
# It's better if your service is up and running but not throttling API, then to have it throw errors for all users
# Make sure you monitor your redis-server so that it's never down. monit is a great tool for that.
end
end
@app.call(env)
end
def bad_request
body_text = "Bad Request"
[ 400, { 'Content-Type' => 'text/plain', 'Content-Length' => body_text.size.to_s }, [body_text] ]
end
def over_rate_limit
body_text = "Over Rate Limit"
[ 503, { 'Content-Type' => 'text/plain', 'Content-Length' => body_text.size.to_s }, [body_text] ]
end
end
To use it on our ‘Hello World’ rack application, simply add it with the use keyword and the :requests_per_hour option:
require 'api_throttling'
use Rack::Lint
use Rack::ShowExceptions
use ApiThrottling, :requests_per_hour => 3
run lambda {|env| [200, {'Content-Type' => 'text/plain', 'Content-Length' => '12'}, ["Hello World!"] ] }
That’s it! Make sure your redis-server is running on port 6379 and try making calls to your api with curl. The first 3 calls will be succesful but the next ones will block because you’ve reached the limit that we’ve set:
$ curl -i http://joe@localhost:3000 HTTP/1.1 200 OK Content-Type: text/plain Content-Length: 12 Connection: keep-alive Server: thin 1.0.0 codename That's What She Said Hello World! $ curl -i http://joe@localhost:3000 HTTP/1.1 200 OK Content-Type: text/plain Content-Length: 12 Connection: keep-alive Server: thin 1.0.0 codename That's What She Said Hello World! $ curl -i http://joe@localhost:3000 HTTP/1.1 200 OK Content-Type: text/plain Content-Length: 12 Connection: keep-alive Server: thin 1.0.0 codename That's What She Said Hello World! $ curl -i http://joe@localhost:3000 HTTP/1.1 503 Service Unavailable Content-Type: text/plain Content-Length: 15 Connection: keep-alive Server: thin 1.0.0 codename That's What She Said Over Rate Limit
The code is up on github if you want to use it.
22 comments so far
Leave a reply
It’s worth noting that the second code sample is missing some values in the second, empty hash on line 4. Specifically, I found that a Content-Type and a Content-Length was required if using Rack::Lint.
The second last code sample however has these necessary options.
Now I just need to learn how to read the article in its entirety before hitting Google to find out what the error messages mean
mlambie,
Thanks for pointing that out. I updated the blog post to reflect that.
If you want to use Redis as Rails/Merb/Sinatra cache store, Rack::Session store or Rack::Cache store, you can check out my gem on GitHub: http://bit.ly/7tUj4
enjoy!
Nice tutorial, Luc!
We are going to implement an export API for our web application where resource limiting is very important. This is going to be very helpful! Thanks
antirez,
I should thank you for Redis, not the other way around
[...] How-To: Rack Middleware for API Throttling – A more reasonable use case for Rack than some I’ve seen. [...]
It might be worth updating the code to add a ‘Retry-After’ header:
“…If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.…”
(from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html)
Will,
That’s a very good idea. I’ll add the header to the code on github.
What should be done with the old key/values?
Brandon,
Obviously, old key values are a waste so you can flush these out via a cron job.Take a a look at the FLUSHDB and FLUSHALL redis commands. If you run via a cron job, make sure you run the job at the begining of the hour (for the limit per hour implementation).
I like Redis but Memcached would probably be a better pick here. It has the same INCR command, without any of the maintenance concerns (i.e. flushing old values) since it will auto-expire keys and does no persistence.
Glad to see some great examples of how to use Rack. Unless I was attending the wrong sessions, I felt RailsConf was very light on solid examples.
[...] http://blog.messagepub.com/2009/05/05/how-to-rack-middleware-for-api-throttling/ : un tutoriel pour faire de l’API throttling grâce à Rake (l’API throttling est une pratique qui consiste à limiter le nombre d’appels qu’un client peut faire à une service web) [...]
This is stupid.
Use the right tool for the right problem. There is no need to include all of this at such a high level.
For example, nginx has the limit_req directive that works perfectly for something like this. There is no need to incur all the additional overhead of rack, redis blah blah blah.
Leather,
Thanks for your comment. I did look at the limit_req module but the documentation was sparse and it didn’t seem to be able to limit connections per user.
If you have used limit_req before, it would be great if you could write about it and share your knowledge since there is so little documentation on it. I would love to see a solution to this problem using limit_req module.
Rack and redis are not really a overhead. All Ruby frameworks use rack it and if you run any modern Ruby web framework, you are already using it heavily + we are using Redis for other stuff to so it is part of our infrastructure. If you do not want to use Redis and have Memcached already running, you could easily adapt this code to use memcached instead (see Mike’s comment earlier).
[...] Rack Middleware for API Throttling I will show you a technique to impose a rate limit (aka API Throttling) on a Ruby Web Service. I will be using Rack middleware so you can use this no matter what Ruby Web Framework you are [...]
@Leather,
Limit_req is great, but it doesn’t have flexibility of a higher level tool like ApiThrottling. Being able to limit on a per-user basis make ApiThrottling a heck of a lot more useful. It’s pretty common with API services to limit requests based on contractual agreements (pay more for a higher guaranteed throughput, etc). This could be built into ApiThrottling without much work.
Also, starting off a comment with “this is stupid” is pretty asinine, especially when you obviously haven’t spent the time to understand what ApiThrottling gives you. Luc has made something really great here, and trolling just makes you look, well, stupid.
FYI Redis supports key expiration.
http://code.google.com/p/redis/wiki/ExpireCommand
INCR then EXPIRE a key.
The nice part about the EXPIRE command is that you get a running rate limiter, as a pending expiration is removed after you change a value.
INCR key, EXPIRE key 3600.
@Leather – not everyone can control the nginx config (e.g. when hosted on Heroku).
@Brian: have you actually gotten this to work?
In my experiments and according the documentation, for volatile keys (expired keys), INCR will overwrite the value, treating it as null; your counter will forever be 1 if you use expire.
incr(‘foo’,1)
# foo = 1 # good
incr(‘foo’,1)
# foo = 2 # good
expire(‘foo’,100)
# foo will expire in 100 seconds
incr(‘foo’,1)
# foo = 1 # bad
See this thread:
http://groups.google.com/group/redis-db/browse_thread/thread/9de9fdb23d9b04c2
Using a cron job for cleanup seems to be a reasonable approach to this problem
You might want to actually authenticate the username given against your datastore. Otherwise, an attacker could push someone else over *their* rate limit.
Nevermind this. I just looked how Rack::Auth::Basic actually works