Variables and Params

Concepts covered: Templating and params

A variable is a key and a value stored in the metadata database for the whole deployment, readable from any task in any DAG. It holds a setting that changes on its own timetable, slower than a run and faster than a deploy: the minimum payout below which a host waits for next week, the addresses a report goes to, the number of days of history to rebuild. Variable.get returns the value as a string, and deserialize_json=True parses a JSON value into a dict or a list.

from airflow.sdk import task, Variable

@task
def sum_payouts():
    min_amount = int(Variable.get("payout_min_amount"))
    report_cfg = Variable.get("payout_report", deserialize_json=True)
    print(min_amount, report_cfg["recipients"])

Variables are set in 3 places. The UI has a Variables page under Admin where a key and a value are typed in. From the command line it is airflow variables set payout_min_amount 20, and airflow variables get payout_min_amount reads it back. The worker's environment is the third place: AIRFLOW_VAR_PAYOUT_MIN_AMOUNT is read as the variable payout_min_amount, which suits values that differ by deployment, though a variable set that way is missing from the UI's list. If a key contains password or secret, its value is masked in the UI and in logs, and the same goes for token and api_key. That makes a variable an acceptable home for a lone API key, though a connection is the better one as soon as a host goes with it.

Params: values a run is given

A param is a setting that belongs to one DAG and can differ per run. The file declares each param with a default, and a scheduled run gets the defaults. A run triggered by hand gets whatever the person triggering it fills in. The UI's trigger form asks for each param in its own field, using the type and description the file gave it. region in the DAG below defaults to all, so the nightly run pays every host, and a rerun for one country is a hand trigger with region set to fr.

from datetime import datetime, timedelta
from airflow.sdk import dag, task, Param, Variable

@dag(
    schedule=timedelta(days=1),
    start_date=datetime(2026, 3, 1),
    params={
        "region": Param("all", type="string", description="2-letter code or all"),
        "dry_run": Param(False, type="boolean"),
    },
)
def host_payouts():

    @task
    def sum_payouts(params=None):
        min_amount = int(Variable.get("payout_min_amount"))
        print(params["region"], params["dry_run"], min_amount)

    sum_payouts()

host_payouts()

params arrives in the task the same way ti did, as a parameter the function declares, and holds this run's values as a dict. Param gives each setting a type, so a trigger form that receives dry_run as the word yes is refused before any run is created, and a plain value such as "all" works where no validation is needed. From the command line the same trigger is airflow dags trigger host_payouts --conf '{"region": "fr", "dry_run": true}', and the values pass through the same validation.

On the afternoon of 2026-03-03 an analyst set payout_min_amount to 25 in the UI, and the last row shows the consequence. The 2026-03-02 run's task ran at midnight at the start of the 3rd and read 20; the 2026-03-03 run's task ran at midnight on the 4th and read 25. The variable keeps no history, so the value a run used is in its log and nowhere else. And if the 2026-03-01 run's sum_payouts is cleared on the 5th and reruns, it reads 25, the current value, because Variable.get executes when the try runs with no memory of what an earlier try read. Params behave in 2 different ways on a rerun. The values a person gives at trigger time are stored with the run when it is created, so a cleared task in the fr run still sees fr and true a month later. A scheduled run is given no values, so it stores none, and each try reads the defaults in the DAG file as the file stands then. Change the file's region default from all to us on the 5th, clear the 2026-03-01 run's sum_payouts, and the rerun's log shows us. Which of the 2 a cleared task sees is decided by how its run was started, and the run_id's manual__ or scheduled__ prefix tells you which.

Variable

• One value for the whole deployment
• Set from the UI or CLI; an environment variable works too
• Read when the task runs, so a rerun sees the current value
• For settings that drift: thresholds, recipients, switches

Param

• Declared per DAG, each with a default
• Set per run by the trigger form or --conf
• Trigger values are stored with the run; a scheduled rerun reads the file's defaults
• For choices made at trigger time: a region, a dry run, a date to redo

About this lesson

This section is part of the Tasks and Data: Beginner lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.