The Meal Kit That Knows You

A medium Pipeline Design interview practice problem on DataDriven. Design the pipeline on the canvas: sources, transforms, storage and checks, then get feedback on it.

Domain

Pipeline Design

Difficulty

medium

Seniority

L6

Problem

What turns a meal-kit service's orders and weekly menus into a list every customer, new or returning, can actually order and eat the moment the app opens, when recipes sell out mid-evening and opens peak near 500 per second on Sunday evenings? Design the pipeline that learns from past orders and serves that list to the app.

What they ordered says a lot about what they want next.

Business requirements

• When a customer opens the app, the recommendation list has to load right away or they'll bounce.
• About 15% of users each day are first-time customers with no order history; they still need a useful list, not an empty screen.
• Sold-out items can never appear, dietary restrictions have to be respected, and new recipes get a launch boost.
• Today there's no feedback loop; the model never learns from which recommendations customers actually clicked, so quality can't improve.

Worked solution and explanation

What this really is

This is a feature-store problem dressed up as recipe personalization. The skill being probed is splitting the work across two clocks: features computed ahead of time on a streaming path and a batch path, and a request path that only looks features up and scores them. Most candidates draw a service that joins order history to the menu when the app opens. That design hangs at the dinner-hour peak, shows first-time customers an empty screen, and recommends a sold-out kit, because nothing sits between the model and the customer.

Step 1: Move feature work off the request

streaming_features keeps recent-order signals fresh in online_feature_store; batch_features recomputes the heavy history hourly. At app open, recommendation_service does a key lookup plus a model call. Any join at request time is the part that melts at peak.

Step 2: Give the lookup miss somewhere to land

About 15% of daily opens have no user row. The service treats the miss as a branch, not an error, and reads cold_start_lists: popular-this-week lists per dietary segment, rebuilt by the batch job.

Step 3: Put sold_out_and_dietary_rules after scoring

The model ranks for relevance; the rules guarantee deliverability. sold_out_and_dietary_rules reads live availability from menu_inventory, drops sold-out items, enforces dietary filters, and applies the launch boost. Rules before scoring would force retraining every time inventory changes.

Step 4: Log what was shown, then what was clicked

sold_out_and_dietary_rules writes every impression to feedback_log with its final rank, and app writes the clicks back to the same log. Orders already arrive through order_events. All of it lands in offline_lakehouse and feeds training_job. Clicks without impressions give you no negatives to learn from.

Request-time join

Joins order history to the menu per open. Latency grows with history and traffic. New users join to nothing.

Pre-computed lookup

Reads one row from online_feature_store. Latency is flat. A miss routes to cold_start_lists.

Common follow-up questions

• A new recipe has zero engagement. How does it get enough exposure to learn from without flooding lists? (Launch boost as a bounded `sold_out_and_dietary_rules` parameter, with `feedback_log` measuring the result.)
• An item sells out mid-session. Where would a delay in hiding it show up? (The freshness of the availability read from `menu_inventory` into `sold_out_and_dietary_rules`, not the model.)