🧑💻 Online code judge system design
A simplified, healthy design: 4 app servers behind a load balancer, a cache (80% hit rate), an async write queue, a CDN for static files, a primary database. At 900 requests per second it runs at p95 46 ms with 0.0% errors for about $370/month.
Healthy at 900 req/s
- p95 latency (server)
- 46 ms
- Error rate
- 0.0%
- Est. cost / month
- $370
- excl. bandwidth
App servers
400 req/s each · $40/mo
Hit rate
DB read replicas
+1,250 reads/s each · also applies every write · $120/mo
Primary DB: 1,250 queries/s (1 query per request). How the numbers work
How it works
- Browsing problems is cached reads; submitting code is the expensive part.
- Submissions go into a queue and sandboxed workers run them one by one, so a contest rush cannot crash the site.
- The editor and problem pages are static assets served from the CDN.
Set traffic to 5,000 req/s, like a contest start, and scale it back to healthy.
Use the Build and Traffic tabs above. The overloaded part turns red and starts dropping requests.