ML Data Engineer Interview

Source events (clicks, views, purchases)
   -> Kafka (entity-keyed topics)

REAL-TIME PATH (online features):
   -> Flink stateful job (compute features in flight)
   -> Redis (online store, p99 < 50ms reads)
   -> dual-write to S3 feature log

BATCH PATH (offline features):
   -> S3 raw events (event-time partitioned)
   -> Spark daily batch (compute aggregate features)
   -> S3 feature parquet
   -> Iceberg table for query

FEATURE CATALOG (Feast or in-house):
   -> Single source of truth for feature definition
   -> Feature owners, refresh schedule, SLA, downstream consumers

TRAINING DATA CONSTRUCTION:
   -> Spark as_of_join (feature_ts <= label_ts)
   -> Produces leak-free training rows

ONLINE INFERENCE:
   -> Service reads from Redis by entity_id
   -> On miss: default value or fallback model
   -> Latency-budget enforced at gateway

MONITORING:
   -> Daily PSI / KS-test on feature distributions
   -> Alerts on drift > threshold
   -> Online vs offline reconciliation job (catches divergence)

-- as-of join via window function (Postgres / Snowflake / BigQuery)
WITH ranked AS (
  SELECT
    l.label_id,
    l.user_id,
    l.label_ts,
    l.label_value,
    f.feature_value,
    f.feature_ts,
    ROW_NUMBER() OVER (
      PARTITION BY l.label_id
      ORDER BY f.feature_ts DESC
    ) AS rn
  FROM labels l
  LEFT JOIN feature_log f
    ON f.user_id = l.user_id
    AND f.feature_ts <= l.label_ts
)
SELECT label_id, user_id, label_ts, label_value, feature_value
FROM ranked
WHERE rn = 1;