Computing Foundations › APIs and the Web › Day 27

Day 27: Rate Limits, Pagination, and Error Handling

Day 27 of 365 — Rate Limits, Pagination, and Error Handling

After this lesson you will be able to consume a real API robustly — backing off politely when rate-limited, paginating through large result sets without missing or repeating rows, and classifying failures so you retry only what is safe to retry and never hang or loop forever.

Course
Computing Foundations
Category
APIs and the Web
Reading time
≈ 40 min
Practical time
≈ 30 min
Lesson duration
1h 10m
Last verified
2026-07-12

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-027-rate-limits-pagination-and-error-handling

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/computing-foundations/day-027-rate-limits-pagination-and-error-handling
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters


The idea in plain language

Diagram: the rate-limit headers and what each one reports


Historical background


What it is — and what it is not

Rate limiting is:

It is not:


Why it was created and what problems it solves


How it works

Rate limits and the token bucket

Diagram: how a token bucket rate limiter allows bursts within an average

X-RateLimit-Limit: 60
X-RateLimit-Remaining: 12
X-RateLimit-Reset: 1752345600

Flowchart: exponential backoff with jitter after a 429 response

Pagination styles

Diagram: how insertions shift an offset-paginated window

Diagram: offset pagination versus cursor pagination fetching pages

Errors: classify, then react

Diagram: the order of steps when handling a failed request

OutcomeMeaningRetry?
2xxIt workedNo — you have your answer
4xxYour request was wrongNo — fix it; retrying is pointless
429You are rate-limitedYes — after waiting
5xxThe server failed a valid requestYes — with backoff
Timeout / network failureNo response reached youOnly if idempotent

An everyday analogy

Diagram: the consequences of continuing to send after being rate-limited

A busy telephone helpline.

The helplineThe API
”All our advisors are busy”429
”Please call back in five minutes”Retry-After
Everyone redialling at once at 9amThe thundering herd
Waiting a bit longer each timeExponential backoff
Redialling at a random momentJitter
”Your call is important” and no answerA timeout you must set
Being told your account is closedA 4xx — redialling will not help

Examples in practice

Diagram: the steps of a pagination loop and its stopping conditions

Diagram: fixed backoff compared with backoff plus jitter


Implications: security, privacy, performance, scalability, and cost

Diagram: the timeouts that should be set on an HTTP request


Alternatives: free, open source, and commercial

ToolTypeWhat it gives you
tenacity (Python)Free, open sourceRetries, backoff and jitter as a decorator
urllib3 retriesFree, built inRetry policy configured on the HTTP adapter
Official SDKsUsually freeRetries and pagination already implemented correctly
p-retry, axios-retryFree, open sourceThe same for JavaScript
API gatewaysCommercial, free tiersLimits enforced on your API, not just consumed
Message queuesFree, open sourceNatural pacing and back-pressure for bulk work

TermWhat it actually is
Rate limitA cap on requests per account per window
Token bucketThe refilling budget most limiters implement
BackoffWaiting longer after each successive failure
JitterRandomising that wait to desynchronise clients
Thundering herdMany clients retrying in the same instant
CursorAn opaque marker naming the last row you saw
Idempotency keyA token letting the server recognise a repeat

When to use it — and when not to

Diagram: which failures are safe to retry


The AI connection

Diagram: reactive backoff compared with proactive pacing


Knowledge check

quiz:
  question: "A client retries with exponential backoff but no jitter. After a brief outage, the service comes back and immediately fails again. Why?"
  options:
    - "Every client backed off by the same amount and retried in the same instant, recreating the load"
    - "Exponential backoff waits too long, so requests arrived after the service had already restarted"
    - "The backoff reset the connection pool, forcing every client to reconnect at once"
    - "Without jitter, the retry counter never resets, so clients retried indefinitely"
  answer_index: 0
  explanation: "Deterministic backoff synchronises clients: they all failed at the same moment, all waited the same interval, and all returned together. Jitter exists to break that synchronisation, spreading the retries out so recovery is not immediately undone."

Hands-on exercise

Get rate-limited on purpose, then handle it correctly. Worked through fully in the Day 27 lab directory.

Set API to the base URL of any public API that publishes rate-limit headers — the lab names two that do — then run these against it.

What you are doingCommand
See the limit headerscurl -sI "$API/users" | grep -i ratelimit
Provoke a 429The lab’s local server, or a loop against a permissive test endpoint
Read Retry-Aftercurl -sI URL | grep -i retry-after
Walk pages by offsetcurl -s 'https://jsonplaceholder.typicode.com/posts?_page=1&_limit=5'
Follow a Link headercurl -sI "$API/users?per_page=5" | grep -i link
Set a timeoutcurl --max-time 5 URL
Classify a responsecurl -s -o /dev/null -w '%{http_code}\n' URL

Expected output

x-ratelimit-limit: 60
x-ratelimit-remaining: 57
x-ratelimit-reset: 1752345600
HTTP/1.1 429 Too Many Requests
retry-after: 30
link: <https://api.example/users?since=46>; rel="next"

Validate your work

Troubleshooting

Common mistakes


Practice assignment


Extension challenge


Quiz

Q1. What does an HTTP 429 Too Many Requests response mean?

  1. The server permanently banned your API key
  2. The server crashed while handling your valid request
  3. The URL you requested does not exist
  4. You exceeded the allowed rate; wait before retrying
Show answer

Answer: D. You exceeded the allowed rate; wait before retrying

429 is rate-limit flow control: you are asking too fast for the current window. It is temporary — backing off and retrying later clears it, unlike a permanent ban (usually a 403) or a missing resource (404).

Q2. A server responds with 429 and includes a Retry-After: 30 header. What is the correct client behavior?

  1. Wait 30 seconds, then retry the request
  2. Give up permanently and report a fatal error
  3. Immediately resend the request as fast as possible
  4. Switch to a different URL to avoid the limit
Show answer

Answer: A. Wait 30 seconds, then retry the request

Retry-After tells you exactly how long to wait. When it is present, obey it — you cannot guess a better delay than the server just told you. Only when it is absent do you fall back to exponential backoff.

Q3. Why is jitter (a small random amount) added to exponential backoff delays?

  1. To make the code harder to read
  2. To stop many clients retrying at the same instant
  3. To make each retry wait longer than the last
  4. To guarantee the request eventually succeeds
Show answer

Answer: B. To stop many clients retrying at the same instant

Without jitter, a crowd of clients rate-limited at the same moment all retry at the same moment — the "thundering herd" that recreates the overload. Randomizing each wait spreads them out and lets the server recover.

Q4. Which of these requests is it generally UNSAFE to retry automatically after a timeout?

  1. A GET that reads a list of posts
  2. A PUT that sets a record to a specific value
  3. A POST that creates a new order and charges a card
  4. A DELETE that ensures a record is gone
Show answer

Answer: C. A POST that creates a new order and charges a card

A create-and-charge POST is non-idempotent: the first attempt may have succeeded even though the response never arrived, so a blind retry can create two orders. GET, PUT, and DELETE are idempotent — repeating them has the same effect as doing them once.

Q5. What is the key difference between offset/limit pagination and cursor-based pagination?

  1. Offset returns JSON while cursor returns plain text
  2. Offset pagination is encrypted while cursor pagination is not
  3. Cursor pagination can only ever return one page
  4. Offset shifts under inserts; a cursor anchors to the last row
Show answer

Answer: D. Offset shifts under inserts; a cursor anchors to the last row

Offset says "skip N rows"; if rows are inserted or deleted between requests, the window shifts and you can repeat or skip items, and deep pages get slow. A cursor names a specific row, so new inserts do not move your position, and it stays fast even far into a large set.

Q6. How should a resilient client treat a 4xx client error such as 404 Not Found?

  1. Not retry — the request itself is wrong, so fix it
  2. Retry it repeatedly with exponential backoff
  3. Wait for a Retry-After header and obey it
  4. Treat it exactly the same as a 500 server error
Show answer

Answer: A. Not retry — the request itself is wrong, so fix it

A 4xx means the problem is on your side (bad URL, auth, or body). The identical request will fail identically, so retrying is pointless. The exception is 429, a special 4xx that does clear after waiting.

Q7. Why must a retry loop always have a hard cap on the number of attempts?

  1. Because servers charge extra for the first attempt
  2. Without a cap, a permanent failure loops forever
  3. Because caps make requests faster
  4. Because the HTTP specification forbids more than one attempt
Show answer

Answer: B. Without a cap, a permanent failure loops forever

"Keep trying until it works" becomes an infinite loop the moment the failure is permanent (a 404, a bad key, a down service). A bounded number of attempts guarantees the client gives up gracefully instead of hanging forever.

Q8. What does graceful degradation mean when an API call ultimately cannot succeed?

  1. Crashing the whole program so the error is obvious
  2. Silently pretending the call succeeded and returning empty data
  3. Doing something sensible: a cached copy, partial result, or error
  4. Retrying forever until the server comes back
Show answer

Answer: C. Doing something sensible: a cached copy, partial result, or error

Graceful degradation means failing usefully: fall back to cached or partial data, or report an honest error, rather than crashing or — worse — silently pretending nothing went wrong and returning misleading results.

Glossary

rate limit
A cap a server sets on how many requests a client may make in a window of time, protecting the shared service from overload and enforcing fair (or metered) access.
429 Too Many Requests
The HTTP status code a server returns when you have exceeded its rate limit; it is temporary and clears once you wait and retry.
Retry-After
An HTTP header, often sent with a 429 or 503, telling the client exactly how long to wait before retrying — either a number of seconds or an HTTP date.
X-RateLimit headers
A common convention of response headers (Limit, Remaining, Reset) that tell a client its quota, how many requests are left, and when the window resets.
token bucket
A rate-limiting model in which each request removes a token from a bucket that refills at a steady rate; it permits short bursts up to the bucket size while enforcing a long-run average.
exponential backoff
A retry strategy that waits a delay which doubles after each failed attempt (1s, 2s, 4s, ...) up to a cap, giving an overloaded server room to recover.
jitter
A small random amount added to each backoff delay so that many clients do not all retry at the same instant and stampede the server (the "thundering herd").
pagination
Splitting a large result set into ordered, individually requestable chunks (pages) so no single response has to carry all of it.
offset
A pagination parameter telling the server how many rows to skip before returning the next batch; simple and jumpable but can shift under inserts and slow down on deep pages.
cursor
An opaque marker naming the last row a client saw, used in cursor-based pagination to fetch the next batch stably even when rows are inserted or deleted.
timeout
A deadline a client sets on a request so that a call which never answers becomes a handled failure rather than an indefinite hang.
idempotent retry
Re-attempting a request that is safe to repeat because doing it twice has the same effect as doing it once (as with GET, PUT, and DELETE); non-idempotent writes need an idempotency key or must not be retried blindly.
graceful degradation
Failing usefully when a request cannot succeed — serving a cached or partial result, or returning a clear error — instead of crashing or silently returning misleading data.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.