What Late Data Means

Concepts covered: paLateData, paEventTimeVsProcessingTime

Schema drift is one half of the lesson. The other half is late data. Events do not always arrive in the order they were produced. A click happens on a phone with patchy reception on Tuesday morning, the SDK queues the event locally, and the event is uploaded Thursday afternoon when the phone reconnects to wifi. The event is timestamped Tuesday. It arrives Thursday. Every batch and streaming system in the industry has to decide what to do with that event. The Two Timestamps That Matter These three timestamps are usually within seconds of each other. When they diverge, the gap is what people call lateness. An event with event_time = Tuesday 09:14 and ingestion_time = Thursday 14:27 is two days late. The pipeline that aggregates daily totals has a choice: include this Tuesday event in the Tue

About This Interactive Section

This section is part of the Schema Evolution and Late Data: Beginner lesson on DataDriven, a free data engineering interview prep platform. Each section includes explanations, worked examples, and hands-on code challenges that execute in real time. SQL queries run against a live PostgreSQL database. Python runs in a sandboxed Docker container. Data modeling problems validate against interactive schema canvases. All content is framed around what data engineering interviewers actually test at companies like Meta, Google, Amazon, Netflix, Stripe, and Databricks.

How DataDriven Lessons Work

DataDriven combines four interview rounds (SQL, Python, Data Modeling, Pipeline Architecture) with adaptive difficulty and spaced repetition. Easy problems get harder as you improve. Weak concepts resurface until you master them. Your readiness score tracks progress across every topic interviewers test. Every lesson section ends with problems you solve by writing and running real code, not by picking multiple-choice answers.