How Would You Partition?

Concepts: paPartitioning

Partitioning is how you turn a 10 TB table scan into a 50 GB targeted read. The interviewer wants to hear your thought process for choosing a partition key - not just "partition by date." Choosing a Partition Key Start with how the data is queried. If 95% of queries filter on event_date, that's your partition key. If analysts always filter by region first, consider region. The goal is pruning: the query engine should eliminate partitions before reading any data. A table partitioned by date with 3 years of daily partitions has ~1,095 partitions - a query for one day reads 0.09% of the data. Cardinality Analysis The partition key's cardinality determines how many directories you create. Too few partitions (e.g., by continent - 7 values) means each partition is massive and you get minim