A platform team gets a $48,000 monthly Snowflake bill with no breakdown by pipeline

A medium Pipeline Design interview practice problem on DataDriven. Write and execute real pipeline design code with instant grading.

Domain

Pipeline Design

Difficulty

medium

Problem

A platform team gets a $48,000 monthly Snowflake bill with no breakdown by pipeline. Nobody can say which pipelines should be optimized, which should be retired, or which are growing fastest. The section's pattern: thread a stable identifier through every query the warehouse runs (Snowflake QUERY_TAG, BigQuery job labels, Databricks tags, Redshift query groups). The minimum tag set is pipeline_id and run_id; team is useful for chargeback. Tag the query by replacing the dbt build transform with one whose name states the QUERY_TAG it sets at session start, and add a cost-rollup destination that reads QUERY_HISTORY and groups credits by pipeline.

Practice This Problem

Solve this Pipeline Design problem with real code execution. DataDriven runs your solution and grades it automatically.