# An hourly Spark job writes into a daily-partitioned plain Snowflake table with INSERT INTO

Canonical URL: <https://datadriven.io/problems/an-hourly-spark-job-writes-into-a-daily-partitioned-plain-sn-17bbce2f>

Domain: Pipeline Design · Difficulty: medium

## Problem

An hourly Spark job writes into a daily-partitioned plain Snowflake table with INSERT INTO. The section just named this exact mismatch: when run cadence is finer than partition grain, each retry appends on top and a partial-failure retry obliterates the day's accumulated work. Apply both halves of the section's recipe: an ACID destination and an idempotent intermediate step so the chain is safe end to end.

## Related

- [All practice problems](https://datadriven.io/problems)
- [Mock interview mode](https://datadriven.io/interview/an-hourly-spark-job-writes-into-a-daily-partitioned-plain-sn-17bbce2f)
- [System Design Interview Questions](https://datadriven.io/data-engineering-system-design)
- [Data Engineering Interview Prep Guide](https://datadriven.io/data-engineer-interview-prep)
- [Daily Challenge](https://datadriven.io/daily)

---

Source: DataDriven (https://datadriven.io). 100% free data engineering interview prep. Live code execution against Postgres 16, Python 3.11, and Spark sandboxes. No paywall, no premium tier, no signup gate.