Feature Store Debug: The Skew

ML team's fraud model trains at 0.84 AUC offline and tanks online. We isolated it to feature user_orders_last_24h , offline backfills compute it correctly, but the online value at scoring time is consistently higher. Run tests/test_feature_parity.py to see the divergence on a known prediction. Find the time-leak in the offline pipeline (features/orders_24h.py) and fix it. Defend your fix: what's the upper bound on staleness you've now committed the team to?

features/orders_24h.py

# features/orders_24h.py
# Computes user_orders_last_24h. Offline backfill version (left).
# Online lookup is in features/online.py , read-only.

from datetime import datetime, timedelta
from typing import List, Dict
from features.store import scan_orders


def compute_offline(user_id: str, as_of: datetime) -> int:
    """Number of orders user_id placed in the 24h window leading up to as_of.
    Used by the offline training feature backfill."""
    since = as_of - timedelta(hours=24)
    rows = scan_orders(user_id=user_id)
    return sum(1 for r in rows if since <= r['order_time'] <= as_of)

# features/orders_24h.py
# Computes user_orders_last_24h. Offline backfill version (left).
# Online lookup is in features/online.py , read-only.

from datetime import datetime, timedelta
from typing import List, Dict
from features.store import scan_orders


def compute_offline(user_id: str, as_of: datetime) -> int:
    """Number of orders user_id placed in the 24h window leading up to as_of.
    Used by the offline training feature backfill."""
    since = as_of - timedelta(hours=24)
    rows = scan_orders(user_id=user_id)
    return sum(1 for r in rows if since <= r['order_time'] <= as_of)

Active Now|Sr. Data Engineer (L5)|||4.9k Attempts|1.4k Solves|