AdvancedTechnical
5 min
Scaling API Gateways
API GatewayScalabilityNetworking
Advertisement
Interview Question
What considerations would you make when scaling an API gateway for millions of requests per second?
Key Points to Cover
- Horizontally scale gateway instances behind load balancers
- Enable caching, compression, and connection pooling
- Use async/non-blocking IO for throughput
- Apply rate limiting, circuit breakers, and retries
- Ensure observability of latency and errors
Evaluation Rubric
Explains horizontal scaling and LB use30% weight
Optimizes throughput/latency30% weight
Adds reliability features (circuit breakers)20% weight
Mentions monitoring metrics20% weight
Hints
- 💡Consider Envoy, NGINX, Kong at scale.
Potential Follow-up Questions
- ❓What about multi-region API gateway design?
- ❓How do you prevent cascading failures?
Advertisement