A pipeline consumes from a Kafka click_events topic
A medium Pipeline Design mock interview question on DataDriven. Practice with AI-powered feedback, real code execution, and a hire/no-hire decision.
- Domain
- Pipeline Design
- Difficulty
- medium
Interview Prompt
A pipeline consumes from a Kafka click_events topic. The current consumer has two cardinal sins: it commits the offset before writing to the raw zone (a crash between commit and write loses the message silently), and a webhook handler from a SaaS partner writes directly to the same downstream without a buffer (peak load knocks the consumer over). The section's rules: write first, commit after; route webhook events through a queue. Place the consumer by replacing the Kafka consumer transform with one whose name states 'write-first, commit-after' and adding a queue between the webhook source and the writer so the webhook handler returns 2xx as soon as the queue accepts the event.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write your pipeline design solution with real code execution
- Get instant feedback and a hire/no-hire decision