IntermediateScenario
10 min
Flaky Integration Tests Blocking Releases
CI/CDTestingReliability
Advertisement
Interview Question
CI pipelines are blocked by flaky integration tests. How do you triage and stabilize pipelines?
Key Points to Cover
- Identify flaky tests with historical run data
- Isolate infrastructure vs code test flakiness
- Add retries with quarantine for known flaky tests
- Fix root causes (timeouts, race conditions)
- Parallelize and optimize test environments
Evaluation Rubric
Identifies flaky tests systematically30% weight
Separates infra vs code issues30% weight
Proposes retries/quarantine and fixes20% weight
Stabilizes pipelines effectively20% weight
Hints
- 💡Retry storm can hide real failures.
Potential Follow-up Questions
- ❓How to track flaky tests over time?
- ❓Should flaky tests block releases?
Advertisement