End-to-End Throughput Analysis

A pipeline's throughput is the minimum of its stages' capacities, and the rest of the analysis follows from finding that minimum under each condition the interviewer names: steady peak, a spike, a degraded dependency, and recovery. The same stage is rarely the minimum in all 4.

Little's Law at each hop

Little's Law ties 3 quantities together for any stable stage: items in flight equal throughput times the time each item spends inside. It sizes concurrency, and it exposes memory limits that a throughput check misses.

The last row shows a break the capacity table missed. Each writer keeps up with its share, but holds 800 MB just before its 60-second flush. The writer rows show that peak, arrival rate times interval; Little's Law's 30-second average wait gives half, but memory must hold the peak. If the writers run with 1 GB of memory, the doubling leaves 20% headroom, and a slow commit that stretches the interval to 80 seconds tips them over. A 30-second interval halves the memory and doubles the files written.

Recovery is a separate calculation

Take the outage from the interview: the consumer group stops for 30 minutes at doubled peak, so the backlog is 80,000 events per second times 1,800 seconds. Draining it needs capacity above the live input, and the drain rate is the smallest stage capacity minus the 80,000 still arriving. Consumers are only one of the stages in that minimum.

• 144M

The cache alone changes nothing about recovery, because the writers share the same 90,000 ceiling. Adding 3 writers raises that ceiling to 135,000 events per second, still the narrowest stage, and cuts the drain from 4 hours to about 44 minutes. The profile service then sees only 27,000 calls per second. This is the reasoning the question tests: the fix for steady state and the fix for recovery sit in different stages, and an answer has to find both.

A degraded dependency

If profile calls slow from 4 ms to 40 ms, the cached design still answers 80% of lookups locally. The average cost rises to 8.08 ms (0.8 x 0.1 + 0.2 x 40), a consumer handles around 1,238 events per second, and 48 consumers reach roughly 59,400, below the doubled input of 80,000. With half the pings shed at the gateway, input falls to 56,000 and the fleet holds at 94%. The cache absorbs most of the slowdown and shedding covers the rest, so the design keeps both.

Measuring it in production

The same arithmetic runs against real metrics. The query below reads per-partition samples from a consumer_lag_metrics table, takes the latest sample for each partition, and estimates minutes to drain from the current surplus.

WITH latest AS (
  SELECT
    partition_id,
    end_offset - committed_offset AS lag_events,
    produced_per_sec,
    consumed_per_sec,
    ROW_NUMBER() OVER (
      PARTITION BY partition_id
      ORDER BY sample_ts DESC
    ) AS rn
  FROM consumer_lag_metrics
  WHERE consumer_group = 'trip-enrichment'
    AND sample_ts > current_timestamp - INTERVAL '10' MINUTE
)
SELECT
  partition_id,
  lag_events,
  consumed_per_sec - produced_per_sec AS surplus_per_sec,
  CASE
    WHEN consumed_per_sec <= produced_per_sec THEN NULL
    ELSE CAST(lag_events AS DOUBLE) / (consumed_per_sec - produced_per_sec) / 60
  END AS minutes_to_drain
FROM latest
WHERE rn = 1
ORDER BY lag_events DESC
LIMIT 10

A NULL drain time marks a partition that is falling behind. The shape of the top 10 rows is the diagnosis: lag spread evenly across partitions means the fleet is short of capacity, while 1 or 2 partitions far above the rest point to a hot key that more consumers will not fix.

The answer, in the order you would say it

• Demand: Doubled peak, 60% pings and 40% lifecycle events, lasting about 3 hours.
• First break: Lag age grows a quarter second per second and breaks the 5-minute target in 20 minutes.
• Steady-state fix: A profile cache cuts per-event cost; 96 partitions leave room to scale consumers without repartitioning.
• Next ceiling: Writer memory reaches 800 MB per writer, and 6 writers cap any drain at 10,000 events/s of surplus.
• Recovery: The backlog from 30 minutes down drains at 55,000 events/s of surplus, limited by the 9 writers.
• Overflow policy: Sample pings at the gateway when lag age passes 3 minutes; never drop lifecycle events.
• Cost: Between 12 and 48 consumers on lag signals, capped by the writer and profile quotas.

About this lesson

This section is part of the Backpressure and Scaling: Advanced lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.