Rate limiting

Rate limiting is a server deliberately refusing requests because they are arriving too quickly. Rather than slowing down, most servers reject the extra requests outright and answer with the HTTP status code 429 Too Many Requests.

Limits are usually enforced by a CDN, firewall, or hosting platform rather than the website software itself, so they can apply to a whole domain at once and can change without warning. A response may include a Retry-After header telling the caller how long to wait before trying again.

Rate limiting is aimed at abusive traffic, but it does not distinguish intent - a legitimate crawler reading many pages in a short window looks much the same as an attack. This is why a page that loads instantly in your browser can still be refused during automated testing.

Learn more

• 429 Too Many Requests - MDN's reference for the status code
• RFC 6585: Additional HTTP Status Codes - the specification that defines 429
• Retry-After - the header servers use to ask for a delay
• Crawler