Quality SLAs vs Ops SLAs

Concepts covered: Data Quality & Validation

An SLA states a commitment. The pipeline-as-product framing from Lesson 1 introduced two SLAs as elements of the contract: freshness SLA and quality SLA. They are commonly conflated. They are different commitments to different things, with different consequences when they fail. A pipeline that meets its operational SLA can fail its quality SLA in green. A pipeline that meets its quality SLA can miss its operational SLA without affecting correctness. The producer who treats both as one number ends up over-promising on one and under-detecting failure of the other. The conflation has visible consequences. Status pages that report a single uptime number describe operational SLA exclusively, leaving consumers with no way to distinguish 'late but correct' from 'on time but wrong'. Incident reviews that do not separate the two end up with action items that improve one without addressing the other. The split SLA is more honest, and honesty in producer commitments is the foundation of the trust that makes data products usable.

The Two SLAs

Property	Operational SLA	Quality SLA
Question answered	Did the data arrive on time	Was the data correct
Typical statement	Pipeline finishes by 6am every day	Row count within 50 to 200 percent of baseline; null rate below 1 percent
Failure mode	Late or missing run	Wrong numbers in a successfully-completed run
Detected by	Orchestrator monitoring; missed schedule alerts	Quality gates inside the pipeline
Consumer impact	Dashboard or model shows yesterday's data	Dashboard or model shows wrong data

Why Conflating Them Hurts

An operational SLA of 'fresh by 6am' tells the consumer when to expect the data. A quality SLA of 'correct row counts by 6am' tells the consumer when to expect the data to be both fresh and right. The two are independent. A pipeline can meet 'fresh by 6am' with a 30 percent row count drop. A pipeline can have a flawless row count and miss the 6am deadline because the warehouse was slow. Consumers who hear 'the team has a 6am SLA' assume both meanings are guaranteed. Producers who state a 6am SLA often mean only operational. The conversation has to specify which one, or both.

Stating Both Explicitly

# Both SLAs stated separately in the contract
guarantees:
  operational_sla:
    statement: 'fct_orders updated by 06:00 Pacific each day'
    measurement: 'orchestrator-reported finish time'
    target: 99.0
    window: 'rolling 30 days'

  quality_sla:
    statement: 'all five-pillar gates pass on the run that satisfies the operational SLA'
    measurement: 'count of successful runs with all gates green / total runs'
    target: 99.5
    window: 'rolling 30 days'

  combined_sla:
    statement: 'fresh AND correct by 06:00 Pacific'
    target: 98.5

What The Combined SLA Actually Costs

The combined SLA is the multiplication of the two. A 99.0 percent operational SLA and a 99.5 percent quality SLA produce a combined SLA of 98.5 percent at best. Promising both individually at 99.0 percent and treating that as the joint guarantee is mathematically wrong. The cost compounds further when consumer behavior is sensitive to the combined number: a model that retrains daily on potentially-stale or potentially-wrong data needs to be designed to tolerate the combined error rate, not either component alone.

Operational	Quality	Combined (Best Case)
99.0%	99.0%	98.0%
99.5%	99.5%	99.0%
99.9%	99.9%	99.8%
99.99%	99.99%	99.98%

Designing around each SLA requires different investments. The operational SLA is improved by orchestration investments: warm pools, retry policies, redundant scheduling, removing single points of failure in compute. The quality SLA is improved by suite investments: more pillars covered, tighter thresholds tuned against history, contracts that prevent shape failures from shipping. The two require different teams in some organizations and different budgets in most. Treating them as one budget produces under-investment in whichever one is currently considered solved.

Operational SLA: Pipeline finishes by a stated deadline. Measured by orchestrator finish time. Improved by warm pools, retry budgets, and redundant scheduling.
Quality SLA: Five-pillar gates pass on the run. Measured by green-gate run rate. Improved by suite coverage, tuned thresholds, and contracts in CI.
Combined SLA: The multiplication of the two. The honest number to publish on a status page; the geometric impossibility to avoid promising.

Operational SLA Improvements

Warm pools and pre-provisioned compute
Retry budgets with exponential backoff
Redundant orchestrator instances
Critical-path identification and short-circuiting

Quality SLA Improvements

Coverage of all five pillars at every layer boundary
Threshold tuning against historical data
Contracts in CI to prevent shape failures
Lineage to shorten incident response

A real-time fraud detection feature has a tight operational SLA: data fresh within seconds. The quality SLA is also tight, but a failure that produces no result is preferable to a failure that produces wrong results. Operational and quality both matter; correctness wins ties. A monthly close finance pipeline has a relaxed operational SLA but an unforgiving quality SLA: a wrong number in the close requires a refile and a regulatory disclosure. Knowing which dominates for a given consumer is part of the contract.

Do

State operational and quality SLAs as separate commitments in the contract
Compute the combined SLA explicitly; do not promise the geometric impossible
Report all three (operational, quality, combined) on the producer status page

Avoid

Treat 'the pipeline is up' as the only SLA; up-but-wrong is its own failure mode
Promise high quality SLA targets without a five-pillar suite to back them
Allow operational improvements to mask quality regression; budget for both

Operational and quality SLAs are different commitments; combining them under-detects one failure mode.
The combined SLA is the multiplication of the two; promise math that is achievable.

Quality checks live at boundaries: validate on the way in (schema, nulls) and on the way out (row counts, totals). A failing gate stops bad data before it reaches the warehouse.

source (source)
ingest_check (quality gate, schema + null check)
transform (transform)
output_check (quality gate, row-count check)
warehouse (storage)

source -> ingest_check
ingest_check -> transform
transform -> output_check
output_check -> warehouse

About this lesson

This section is part of the Data Quality: Advanced lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.