# A 2014-era streaming media company runs the Lambda content-engagement pipeline this section's worked Canonical URL: Domain: Pipeline Design ยท Difficulty: medium ## Problem A 2014-era streaming media company runs the Lambda content-engagement pipeline this section's worked example walked through. Two codebases (Spark + Storm), two on-call rotations, storage in HDFS plus HBase, occasional 0.4 percent drift between layers. The constraints that motivated Lambda have shifted: streaming engines now offer exactly-once, tiered storage makes long log retention affordable, unified engines mean one codebase. Apply the Lambda-to-Kappa migration this section walked through: add the Kappa replacement path alongside the existing Lambda layers (the migration order keeps both running until the cutover). Specifically add: (1) a Kafka tiered-storage backing in object storage (S3, GCS, or ADLS) so the event log can hold the longest backfill window; (2) a single Flink streaming pipeline that processes events end-to-end with exactly-once semantics; and (3) an Iceberg materialized view on object storage as the single canonical view (Iceberg gives ACID transactions, schema evolution, and time travel; Trino is the canonical serving engine that reads it). Do not delete the existing Lambda nodes; the migration order says they stay running until consumers cut over. ## Related - [All practice problems](https://datadriven.io/problems) - [Mock interview mode](https://datadriven.io/interview/a-2014-era-streaming-media-company-runs-the-lambda-content-e-9bc42ca1) - [System Design Interview Questions](https://datadriven.io/data-engineering-system-design) - [Data Engineering Interview Prep Guide](https://datadriven.io/data-engineer-interview-prep) - [Daily Challenge](https://datadriven.io/daily) --- Source: DataDriven (https://datadriven.io). 100% free data engineering interview prep. Live code execution against Postgres 16, Python 3.11, and Spark sandboxes. No paywall, no premium tier, no signup gate.