dbt vs Airflow, the practitioner's decision
dbt and Airflow aren't competitors. They sit at different layers of the stack and the right pair is more important than the better tool. This guide pushes past the surface into integration patterns, cost math, and interview questions that actually matter.
What this guide actually says
dbt is a transformation framework that schedules itself badly. Airflow is a scheduler that doesn't transform. Treat them as orthogonal, not competing. If you only ever transform data already in your warehouse, you might not need Airflow at all. The 2026 production pattern is: Airflow (or Dagster) handles the WHEN and HOW; dbt handles the WHAT. dbt tests cover schema-shaped drift — not volumetrics, freshness against external sources, or cross-source contracts. Picking Dagster over either is a real option in 2026: it solves asset-graph ergonomics both Airflow and dbt work around, at the cost of a learning curve and smaller operator ecosystem.
Know dbt vs Airflow the way the interviewer who asks it knows it.
The orthogonality that fixes the question
dbt is a compile-time SQL graph plus materializations plus tests plus lineage. It runs anywhere SQL runs. The unit of value is the model, which is just a SELECT statement that dbt wraps in CREATE TABLE AS or MERGE INTO depending on materialization. The graph is built from ref() and source() calls at parse time. The scheduler is whatever invokes dbt build.
Airflow is an imperative DAG of arbitrary Python tasks with operators, sensors, retries, SLAs, scheduling, and a worker pool. The unit of value is the task. The graph is built from explicit dependencies between tasks at DAG definition time. There is no native notion of data, only of work to do.
dbt is the WHAT. Airflow is the WHEN and HOW. The two play together because dbt's WHAT becomes one task in Airflow's WHEN and HOW. If your interview answer doesn't separate those layers, the interviewer already knows you haven't run a real platform.
Side by side
| Concept | dbt | Airflow |
|---|---|---|
| Primary artifact | Compiled SQL graph (manifest.json) | DAG of Python tasks |
| Where logic runs | Inside the warehouse, as SQL the warehouse executes | On Airflow workers, calling whatever you tell it to call |
| Compute cost driver | Warehouse credits | Worker pool + warehouse credits for whatever it triggers |
| Native scheduling | Crude. dbt Cloud has a cron UI. dbt Core has nothing. | First-class. Cron, data intervals, sensors, SLA misses, retries. |
| Failure model | Model fails, downstream skips, manifest records last_run_status | Task fails, retry policy fires, downstream stays queued |
| Branching / conditional | Ref graph only. No conditional fan-out. | Native via BranchPythonOperator and TaskGroups |
| Sensors / external waits | None native. Use external_source freshness as a hack. | Native. S3KeySensor, ExternalTaskSensor, custom sensors. |
| Lineage | Auto-generated from ref() and source(). dbt docs renders it. | Task-level only. No column lineage. OpenLineage adds some. |
| Tests | Schema tests on data, generic and singular tests, freshness on sources | Workflow tests on DAGs. No data tests without extra tooling. |
| Multi-tool fan-in | One target warehouse per project. Multi-project orchestration is hard. | Trivially. Operators for everything, including multi-cloud. |
When you only need dbt
A surprising number of analytics platforms don't need an orchestrator. If you check most of these boxes, defer Airflow.
Single warehouse: Snowflake, BigQuery, Redshift, Databricks SQL, or Postgres. No fan-out across non-SQL compute.
Ingestion handled by a managed ELT vendor (Fivetran, Airbyte, Stitch). dbt only sees data that already landed.
No Python preprocessing in the pipeline. No ML training loop coupled to fresh data. No external API calls in the transformation path.
A daily or hourly cron is enough. No data-interval awareness, no SLA dashboards, no per-task retry policy.
Team size under 10 engineers. Coordination cost of running Airflow exceeds the cost of dbt Cloud or a cron entry.
No multi-tenant warehouse fan-out. One project, one set of targets, one schedule.
When dbt alone falls apart
Signals you've outgrown dbt's native scheduling. None in isolation is a reason to add Airflow. Two or three together usually are.
You need to land data first. Pulling from APIs, decrypting files, normalizing PII, staging from S3 with custom parsing. dbt does none of that.
Fan-out across non-warehouse compute. Spark, Flink, EMR, Glue, Beam. dbt does not run there; Airflow operators do.
Conditional branches. Run the heavy backfill only on Sundays. Skip the marketing models when GA4 export missed. dbt has no native conditional execution.
Multi-warehouse fan-in. Snowflake plus Postgres plus BigQuery in the same pipeline. dbt projects are single-target.
ML pipelines. Feature freshness, training trigger, model registry coupling. dbt models can't promote a model artifact.
Cross-tool sensor logic. Wait for the Salesforce export. Wait for the upstream dbt project run in another team. Wait for an SLA window to open.
Complex SLAs. SLA miss alerting, on-call paging, SLO budget burn alerts. Airflow has SLA semantics. dbt does not.
Five ways teams actually wire dbt and Airflow
Each pattern has a real trade-off. Picking depends on your model count, team shape, and tolerance for cold starts.
Airflow + dbt operator (most common)
Airflow ingests data via custom operators or managed ELT, then triggers a single BashOperator running dbt build. Simple, fast to wire, easy to reason about. Trade-off: one Airflow task fans out into dozens of dbt models, so model-level retries aren't visible in the Airflow UI. A single failing model fails the whole task. Use when you want minimum moving parts and your team already debugs dbt logs natively.
Airflow KubernetesPodOperator + dbt
dbt runs in a fresh pod per task. Cleanest dependency isolation. The dbt image carries its own version of dbt-core, profiles, and Python deps. Airflow workers don't need any dbt dependency installed. Scales across teams sharing an Airflow cluster but each owning their dbt project. Cost: cold start. Pod spin-up adds 10-40 seconds per task. For pipelines with 30+ dbt invocations, that's real. Mitigate with image pinning, warm pools, or batched dbt build per project rather than per model.
dbt Cloud + Airflow webhook
You bought dbt Cloud for the IDE, CI hooks, and job UI. Airflow now has to coordinate dbt Cloud jobs alongside ingestion. Use the dbt Cloud trigger operator (or a thin webhook task) to start a dbt Cloud job, then poll for completion via the dbt Cloud sensor. The catch: two scheduler UIs, two billing models, two places to look when a run is late. Works only if someone owns the question of which scheduler is the source of truth.
Dagster instead of Airflow
Dagster treats every dbt model as a software-defined asset. The dbt graph and the Dagster graph merge. You get asset-level retries, lineage, freshness policies. The dbt run is no longer a black box inside one task. Trade-off: smaller operator ecosystem than Airflow. Migrating from Airflow to Dagster is a quarter of work, not a sprint. Greenfield analytics platforms in 2026 should evaluate Dagster before defaulting to Airflow.
Airflow + Astronomer Cosmos
Cosmos parses the dbt manifest at DAG render time and emits one Airflow task per dbt model. You get model-level UI in Airflow, model-level retries, and the Airflow task tree mirrors the dbt graph. Best of both for shops standardized on Airflow. Watch out: DAG render time grows with model count (Cosmos pre-1.5 stalled at 800+ models), and every dbt model becomes an Airflow task, which can inflate the metadata DB faster than expected. Plan retention.
2026 cost math, honestly
Approximate monthly cost for a mid-size pipeline: ~50 dbt models, 30 ingestion DAGs, daily + hourly runs. Warehouse cost excluded except where noted.
| Option | Mid-size monthly cost | Ops burden | Scheduler latency | Where it breaks |
|---|---|---|---|---|
| dbt Cloud (Team) | ~$100/dev/mo + warehouse | Lowest. dbt Cloud handles UI, scheduling, IDE, CI. | Run starts within a minute of cron | When you outgrow native scheduler: branching, fan-out, cross-tool waits. |
| dbt Core on Airflow EC2 | ~$300-600 in EC2 + self-managed RDS | Highest. You own scheduler, workers, metadata DB, upgrades. | Run starts at next data interval boundary | First Airflow CVE. First worker pool exhaustion. First metadata DB lock. |
| Astronomer Astro (managed Airflow) | ~$1,200-2,500 for 2-4 worker deployment + warehouse | Medium. They run Airflow. You write DAGs. | Same as Airflow | Per-task pricing creeps. Cosmos at high model counts blows DAG render budgets. |
| AWS MWAA | ~$700-1,400 for small environment + warehouse | Medium. AWS runs scheduler and workers. You manage requirements.txt. | Same as Airflow, measurable cold scheduler delay | Plugin compatibility. MWAA pins versions. Some Airflow providers lag. |
| Dagster Cloud (Pro) | ~$10/asset materialization, mid-size pipelines land $400-1,200 | Low. They run the orchestrator. You write assets. | Asset materialization triggers in seconds via sensors | Smaller operator ecosystem. Custom integrations more often DIY. |
Myth vs reality
Myth: dbt replaces Airflow
Reality: dbt schedules itself like a cron. The moment your pipeline needs to wait, fan out, branch, page on-call, or coordinate non-SQL work, you need an orchestrator. dbt is a fantastic transformation tool that does not orchestrate. Stop trying to make it.
Myth: Airflow can do dbt's job with PythonOperator and Jinja
Reality: yes, and you'll regret it within a quarter. You'll rebuild ref(), incremental materialization, snapshots, and tests by hand. You'll lose lineage. Your analytics engineers will quit because the merge conflict surface is someone else's Python code.
Myth: dbt tests = data quality covered
Reality: dbt tests catch null, unique, accepted values, and relationships. They don't catch volumetric drift, semantic regressions, freshness against external systems, or cross-source contract breaks. You still need elementary, re_data, Monte Carlo, or a homegrown freshness layer.
Myth: The Airflow scheduler is the bottleneck at scale
Reality: at most shops the bottleneck is the worker pool config or SLA misconfigurations. The 2.x scheduler is fast. If your DAG runs are queueing, the answer is almost always more worker slots, parallelism tuning, or splitting the metadata DB — not a different orchestrator.
Myth: Just use Dagster
Reality: Dagster solves real ergonomic problems around assets and dbt integration. It also adds a learning curve your team must absorb. Connector breadth is smaller. Hiring familiarity is smaller. Don't migrate without budgeting for both.
Decision matrix
Eight common situations. Starting points; push on the trade-offs from there.
| Situation | Pick | Reason |
|---|---|---|
| Single Snowflake warehouse, only SQL transformations, under 10 engineers | dbt core alone | Cron or dbt Cloud is enough. Airflow is overkill. |
| Multiple data sources, Python ingestion, complex retries | Airflow + dbt | Airflow handles fan-in and retries. dbt owns the warehouse layer. |
| ML training pipelines coupled to feature freshness | Airflow | Sensors, conditional branches, model registry hooks live in Airflow. |
| Want a single tool for ingestion + transformation + assets | Dagster | Asset-based mental model unifies dbt, Python, and Spark in one graph. |
| Already on Airflow, looking at adding dbt | Add dbt with Cosmos or BashOperator | Lowest migration cost. Cosmos when you want model-level UI. |
| Greenfield analytics stack at a startup | dbt + dbt Cloud, defer Airflow | Postpone orchestration complexity until you have non-SQL work. |
| Multi-cloud, Kafka in the loop, 50+ DAGs | Self-managed or MSK + Airflow + dbt | Operator breadth and multi-cloud are Airflow strengths. |
| Need column-level lineage across the stack | Dagster or dbt + OpenLineage on Airflow | Airflow alone is task lineage. Column lineage requires either Dagster assets or OpenLineage. |
Where data modeling enters the picture
dbt is also where most teams encode their dimensional model. The choice between SCD Type 1, Type 2, and snapshots lives inside dbt and ripples through Airflow scheduling.
dbt snapshots are the canonical way to track slowly changing dimensions in dbt. The snapshot block writes valid_from and valid_to to a target table whenever a check or timestamp strategy detects a change. dbt handles merge logic, surrogate key, and valid window maintenance.
The Airflow side is the schedule. Snapshots almost always run on a different cadence than transforms. Snapshots run every hour or six hours to capture change. Transforms run nightly to roll up. Mixing the two on the same dbt invocation hides the cadence mismatch and makes failure modes harder to reason about.
The interview question that exposes whether you've run this in production is volumetrics. How big does the snapshot table get? How do you prune? How do you handle backfills? How do you reconcile a snapshot that started with bad data? None of those questions have a dbt-only answer — all involve the orchestrator.
What interviewers actually grade on
They don't ask which is better. They ask scenarios. Five questions that come up in real loops.
Walk me through your orchestration topology for an analytics platform serving 50 dashboards.
Strong answers name layers explicitly. ELT vendor lands raw. Airflow or Dagster orchestrates. dbt models stage, intermediate, and mart layers. BI tool reads from marts. Then explain who owns what, where freshness alerts live, how a failed source flows downstream, where retry logic sits. Bonus for naming the SLA story per dashboard tier.
Your dbt model takes 4 hours and blocks downstream models. How do you fix it?
First instinct should not be to throw warehouse compute at it. Read the explain plan. Check for full table scans on the source. Convert to incremental with a high-cardinality unique_key. Look for cross joins from accidental fanout. Partition or cluster the source. Last resort, materialize an intermediate stage. Naming the diagnostic order is what interviewers grade.
How would you migrate from Airflow + custom SQL to Airflow + dbt without freezing reporting?
Inventory the SQL. Identify the marts driving the top 20 dashboards. Build dbt models in parallel as materialized=view first, validating against existing tables row-for-row. Cut over by repointing dashboards, not by deleting old tables. Keep the old DAG running in shadow for two weeks. Decommission only after a clean backfill comparison.
What does a late-arriving event do to your dbt incremental models? How do you reconcile?
The is_incremental() filter on event_ts misses the late row. The fix depends on tolerance. For low-volume late events, run a daily reconciliation model that re-processes a 7-day rolling window with a merge strategy. For high-volume, route late events to a separate table and union at query time. Discuss the trade-off between completeness and cost.
When would you reach for Dagster instead of either?
When the asset graph is the natural unit of work. When dbt models, Spark jobs, and Python feature pipelines all need to live in one lineage view. When you want freshness policies on assets, not schedules on DAGs. When you're starting fresh and your team can absorb the learning curve. Honest answer includes the trade-off, not just the sales pitch.
Frequently asked questions
Are dbt and Airflow really not competitors?+
Should I learn dbt or Airflow first?+
Is dbt Core or dbt Cloud the right choice?+
Should I migrate from Airflow to Dagster?+
Can I use Airflow without dbt?+
Can I use dbt without Airflow?+
How important is Airflow knowledge for DE interviews?+
Does Cosmos solve the dbt-model visibility problem in Airflow?+
Drill the patterns interviewers actually test
- 01
Active recall beats re-reading by 50%
Cognitive-science meta-reviews (Dunlosky et al., 2013) rank practice testing as a top-tier study technique, while re-reading and highlighting rank near the bottom
- 02
76% of hiring managers reject on the coding task, not the resume
From HackerRank's 2024 Developer Skills Report. Candidates who look strong on paper still fail the live screen if they haven't done timed, executable practice
- 03
Five problem shapes cover 80% of data engineer loops
Dedup, sessionization, top-N-per-group, slowly-changing dimensions, partition tricks. Writing the shapes by hand turns the unfamiliar into pattern recognition
Adjacent interview prep
More data engineer interview prep guides
Data Engineer vs AE roles, daily work, comp, skills, and which to target.
Data Engineer vs MLE roles, where the boundary lives, comp differences, and how to switch.
Data Engineer vs backend roles, daily work, comp, interview differences, and crossover paths.
When SQL wins, when Python wins, and how Data Engineer roles use both.
Snowflake vs Databricks, interview differences, role differences, and how to choose.
Kafka vs Kinesis, throughput, cost, ops burden, and the Data Engineer interview implications.