How Much Will This Cost?

Concepts covered: Columnar vs Row Formats, Compression & Encoding

The lines on the bill

A storage estimate has 4 lines, and in analytics the last one usually dominates. Storage is billed per GB-month by tier. Requests are billed per 1,000 calls, with list and put costing about 10 times a get. Retrieval from cold tiers is billed per GB restored, on top of a wait. Compute is billed either by the TB a query scans or by the hours a cluster runs, and the scan model prices the format and partition decisions from the first 2 sections directly.

Prices are the published list prices of the largest cloud object store for its US regions in 2026, rounded. Your provider and region differ by tens of percent, and the ratios between tiers are what the model turns on.

Worked model: 1 year of the events table

Scale the events table to the 50 GB of Parquet a day the partition section assumed, which is about 330 GB a day as CSV at the first section's 400 GB to 60 GB ratio. After 1 year the Parquet copy holds 18.25 TB and the CSV copy about 120 TB. Held entirely in the standard tier, month 12 costs 18,250 GB times $0.023, which is $420 for Parquet, and 120,450 GB times $0.023, which is $2,770 for CSV. The format decision alone moved $2,350 a month before a single query ran.

Now apply a lifecycle policy to the Parquet copy: standard for 30 days, infrequent access until day 90, archive after. The 1,500 GB in standard costs $34.50. The 3,000 GB in infrequent access costs $37.50. The 13,750 GB in archive costs $49.50. The month-12 bill drops from $420 to $121.50. The catch comes from the third section's consumer question: a dashboard that compares this quarter to the same quarter last year now touches archived files, which cannot be read in place. Its query fails on them until someone issues a restore, which takes hours, bills per GB restored and produces a temporary copy. That is why the archive transition belongs after the longest query window: if analysts query 13 months back, the archive step moves to day 400, and the bill becomes $34.50 plus 16,750 GB times $0.0125, which is $243.88.

{
  "rules": [
    {
      "prefix": "events/",
      "transitions": [
        {"after_days": 30, "tier": "infrequent_access"},
        {"after_days": 400, "tier": "archive"}
      ],
      "expire_after_days": 2555
    }
  ]
}

Compute: where the format decision gets multiplied

Engines that bill per TB scanned commonly list around $5 per TB. A query that reads the full year of CSV scans about 120 TB and costs about $600. The same query on Parquet without a partition filter scans 18.25 TB and costs $91. Reading only the 3 columns it needs, which hold the same 2.5% of the bytes they did in the first section, it scans about 450 GB and costs about $2.30. Adding the date partition filter cuts that to one day of those columns, about 1.25 GB, for about 0.6 cents. Multiply by the 1,000 queries a day a 400-user dashboard produces and the storage figures above stop mattering: $600,000 a day against $91,000 against $2,300 against $6.25. The interviewer who asks 'how much will this cost' is usually checking whether the candidate reaches that multiplication.

• 1.8x
• 6.4x
• 6.7x

Two smaller lines that bite

Request costs rarely reach dollars but reveal the small-file problem. If the hourly loads from the partition section produced 1 million files over the year, a full scan issues 1 million get requests at about $0.0004 per 1,000, which is $0.40, and spends minutes on footer reads; the money is negligible and the latency is the whole story, which the advanced tier takes up. Time travel from the table format section is a storage line: the 4.5 TB of expired-but-retained versions sits in the standard tier at $103.50 a month, so a retention window is a budget decision as much as a recovery one.

Do

• Convert at landing so the 6.7x size ratio applies to every downstream copy
• Set lifecycle tiers from the longest query window rather than the ingest rate
• Price a query by bytes scanned, and quote it per day at dashboard volumes
• Count retained snapshots as storage you pay for

Avoid

• Quote storage alone when a per-scan engine is in the design
• Archive data a dashboard still reads
• Mistake request dollars for the small-file cost; the cost is latency
• Assume list prices match across providers and regions

About this lesson

This section is part of the The Storage Question: Intermediate lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.