Loading section...

Full Refresh or Incremental?

Concepts: paFullVsIncremental

The interviewer is not asking you to pick one. They are testing whether you can reason about the tradeoff. The trap is saying "incremental, obviously" without acknowledging that every pipeline starts as a full refresh and that full refresh is correct until it is not. The Cost Crossover Your answer should frame this as a cost function. Full refresh cost grows linearly with source table size. Incremental has fixed overhead - change detection, merge logic, state tracking - plus variable cost that grows with change volume, not total data. At some point the lines cross. Senior signal: name the crossover for your use case. Say "At 50M rows with a 2% daily change rate, incremental saves us 4 hours of warehouse compute per run." That level of specificity tells the interviewer you have actually don