Python Mock Interview for Data Engineering

Transforming, cleaning, reshaping, and validating data. This is the core of what data engineers do with Python, and it is the core of what interviewers test. Questions involve parsing messy inputs, handling missing values, deduplication, type conversion, and restructuring nested data into flat formats. Patterns: Flatten nested JSON into tabular rows; Deduplicate records using a composite key with tiebreaker logic; Validate data against a schema and collect all violations (not just the first); Convert between data formats: CSV to JSON, JSON to Parquet, XML to dicts; Handle mixed types in a column (strings that should be integers, dates in 4 different formats). Why it matters: Data manipulation is 40% of questions because it is 80% of the actual job. Every pipeline starts with messy input data. Interviewers want to see that you can handle the mess without breaking, and that your code is readable enough for the next engineer to maintain.

Building extract-transform-load workflows in Python. Questions test your ability to design a pipeline that reads from a source, applies transformations in the correct order, handles errors gracefully, and writes to a destination. The focus is on correctness and resilience, not speed. Patterns: Build a pipeline that retries failed API calls with exponential backoff; Implement idempotent writes: running the pipeline twice produces the same result; Process a large file in chunks to stay within memory limits; Handle late-arriving data by merging it into existing results; Log every transformation step for debugging and auditability. Why it matters: ETL questions separate data engineers from Python developers. A Python developer writes code that works once. A data engineer writes code that works at 3am when the source system sends a file with a new column, a missing header, and timestamps in a different timezone.

Reading and processing CSV, JSON, Parquet, XML, and log files. These questions test your knowledge of Python's I/O libraries, encoding handling, and your ability to deal with malformed input. Real-world data files are never clean, and interviewers know this. Patterns: Parse a CSV with inconsistent quoting and embedded newlines; Read a multi-gigabyte JSON file line by line (JSONL format) without loading it all into memory; Extract structured data from semi-structured log files using regex; Handle UTF-8 encoding errors gracefully (skip, replace, or report); Process a ZIP archive containing multiple CSVs with different schemas. Why it matters: File parsing is a litmus test for production experience. Candidates who have only worked with clean DataFrames struggle with encoding issues, malformed rows, and memory constraints. Interviewers use file parsing questions to find engineers who have debugged real pipelines.

Extracting data from REST APIs with pagination, rate limiting, authentication, and error handling. These questions combine HTTP knowledge with data engineering patterns like incremental extraction and checkpoint management. Patterns: Paginate through an API that uses cursor-based pagination; Implement rate limiting with a token bucket or simple sleep; Handle API errors: retry on 429 and 500, fail on 400 and 403; Extract data incrementally using a high-water mark timestamp; Authenticate with OAuth2 and refresh expired tokens. Why it matters: APIs are the extraction layer of most modern pipelines. Interviewers test whether you can write reliable extraction code that does not crash when the API returns unexpected responses, and that does not get your company's API key revoked by ignoring rate limits.

DataFrame operations: merge, groupby, pivot, melt, apply, and vectorized operations. Pandas questions test whether you can write efficient tabular transformations without falling back to row-by-row iteration. Patterns: Group by multiple columns and compute multiple aggregations in one pass; Merge DataFrames with different column names and handle duplicates; Pivot long data to wide format and fill missing combinations; Replace iterrows() loops with vectorized operations for 100x speedup; Handle categorical data, datetime parsing, and memory optimization with dtypes. Why it matters: Pandas is a polarizing topic. Some DE teams use it heavily; others prefer pure Python or PySpark. When it does appear, interviewers specifically test vectorized thinking. Writing a for loop over DataFrame rows is a red flag that tells the interviewer you do not understand how pandas actually works under the hood.

Before writing any code, explicitly define: What is the input type and shape? What is the expected output type and shape? What are the edge cases? Writing this as a docstring or comment before coding shows the interviewer you think before you type. It also prevents the most common mistake: writing 80% of the solution before realizing you misunderstood the output format.

Interviewers evaluate code organization. A single 60-line function that parses, transforms, validates, and writes data is hard to read and hard to score. Break it into 4 functions of 15 lines each. Name them clearly: parse_input, transform_records, validate_output, write_results. This structure also makes it easier to test and debug during the interview.

A bare except: pass is worse than no error handling at all. It tells the interviewer you are hiding bugs. Instead, catch specific exceptions, log useful context (which file, which row, what the bad value was), and decide whether to skip the bad record or abort the pipeline. DataDriven's evaluator specifically checks for bare except blocks and awards points for targeted exception handling.

If the question mentions 'large file' or 'millions of records,' the interviewer is testing whether you will load everything into memory. Use generators, process in chunks, or stream line by line. Saying 'I would use a generator here because the file might not fit in memory' is worth points even if the test data is small.

Run your code mentally (or actually, on DataDriven) with: empty input, single record, None values, duplicate keys, and the largest input you can imagine. If your code handles all five, you have covered 90% of what the evaluator checks. This habit also impresses interviewers because it shows you think about failure modes proactively.