Where a Config Comes From

Concepts covered: Config resolution

3 places to set one config

materialized can be set in 3 files that all claim the same model. dbt_project.yml sets it for every model under a folder. A properties file sets it for 1 named model. The model file sets it for itself with config(). When they agree there is nothing to decide. When they disagree, dbt follows 1 rule: the setting closest to the model wins.

name: analytics

models:
  analytics:
    +materialized: view
    marts:
      +materialized: table

The + marks a config key as opposed to a folder name, and the nesting follows the folder tree: marts here means models/marts/. A deeper key overrides a shallower one, so every model under models/marts/ is a table and every other model in the project is a view, before any other file has spoken.

models:
  - name: mr_by_state
    config:
      materialized: view
  - name: mr_by_project
    config:
      materialized: view

{{ config(materialized='table') }}

select
    project_id,
    count(*) as mrs
from {{ ref('stg_merge_requests') }}
group by project_id

Resolving 3 models

Read each row from right to left and stop at the first value. The staging model is set in 1 place, so it's a view. mr_by_project is a table by folder, but its properties name it a view, and the properties sit closer to the model than the folder rule does, so it is a view. mr_by_state has all 3, and the config() call inside the file beats both: a table. A folder default is a statement about what most models there should be; the 2 closer levels are for the exceptions.

Deeper folders, moved files

name: analytics

models:
  analytics:
    +materialized: view
    marts:
      +materialized: table
      finance:
        +materialized: view

Folder rules nest as deep as the folders do, and among them the deepest key whose folder contains the model wins. A model in models/marts/finance/ is a view, one directly in models/marts/ is a table, and one in models/staging/ falls back to the project-wide view. Because a folder rule follows the file's location, moving a file can change how it's built without changing a line inside it. Move a model with no config() and no properties entry from models/staging/ into models/marts/, and the next run builds a table where a view stood.

Seeing the answer dbt reached

target/run shows which setting won. A statement that opens with create table or create view is the materialization dbt resolved, whatever the 3 files seem to say, so the habit from section 1 covers configs as well as SQL. One config behaves differently: tags from every level are combined rather than replaced, so a model tagged nightly by its folder and finance by its properties carries both.

Materialized and tags follow different rules on the same model. A materialization has exactly 1 answer, so the closest level replaces the others. Tags are labels, and a model can hold any number of them, so each level adds its own.

Do

• Set the common case once, in dbt_project.yml, by folder
• Use a properties file or config() only for the models that differ
• Check target/run when you are unsure which level won
• Keep the 3 levels in agreement where you can, so nobody has to resolve them

Avoid

• Repeat the folder default inside every model file
• Set the same config in 2 places with 2 values and leave it for the next reader
• Assume a folder rule applies to a model whose file says otherwise
• Guess at the result when dbt has already written it to disk

About this lesson

This section is part of the Jinja and Macros: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.