IntermediateScenario
10 min
Log Ingestion Failure
LoggingMonitoringTroubleshooting
Advertisement
Interview Question
Your centralized logging pipeline stops ingesting logs from multiple services. How do you debug?
Key Points to Cover
- Check log forwarders/agents for errors
- Validate ingestion pipeline (Kafka/Fluentd/Logstash)
- Check disk/network saturation on collectors
- Inspect schema/parsing errors in pipeline
- Add redundancy and retry mechanisms
Evaluation Rubric
Validates forwarder/agent status30% weight
Analyzes ingestion pipeline30% weight
Considers infra/disk/network issues20% weight
Suggests redundancy/retries20% weight
Hints
- 💡Check for dropped events due to parsing errors.
Common Pitfalls to Avoid
- ⚠️Focusing solely on the central system and neglecting the distributed agents.
- ⚠️Not having proper logging or monitoring for the pipeline components themselves.
- ⚠️Assuming network connectivity is stable without verification.
- ⚠️Ignoring resource constraints on any part of the pipeline, especially the ingestion points or the final storage.
- ⚠️Making changes without proper rollback plans or thorough testing.
Potential Follow-up Questions
- ❓How do you prevent log pipeline overload?
- ❓What about log sampling?
Advertisement