Compaction Strategies

Compaction is the process of merging many small files into fewer large files. It is a background maintenance job, not part of the main pipeline. The interview tests whether you know how to design a compaction job with the right target file size, scheduling, and partition awareness. Compaction Design The Follow-Up Trap The strong-hire detail: 'I would add a compaction monitoring metric: average file size per partition. When average file size drops below 64 MB, the compaction job triggers automatically. This self-healing approach prevents small file buildup from ever reaching the point where queries degrade.'