🔎 Search autocomplete system design
A simplified, healthy design: 18 app servers behind a load balancer, a cache (90% hit rate), a primary database. At 5,000 requests per second it runs at p95 58 ms with 0.0% errors for about $890/month.
Healthy at 5,000 req/s
- p95 latency (server)
- 58 ms
- Error rate
- 0.0%
- Est. cost / month
- $890
- excl. bandwidth
App servers
400 req/s each · $40/mo
Hit rate
DB read replicas
+1,250 reads/s each · also applies every write · $120/mo
Primary DB: 1,250 queries/s (1 query per request). How the numbers work
How it works
- Every keystroke is a request, and it must answer in well under 100 ms to feel instant.
- The same prefixes ("how to", "wea") are typed by millions, so suggestions are almost always cached.
- Suggestion lists are rebuilt offline in the background, not on every request.
Lower the cache hit rate to 50% and watch p95 latency climb.
Use the Build and Traffic tabs above. The overloaded part turns red and starts dropping requests.