Twitter (X) Data Engineer Interview

from collections import deque
from datetime import datetime, timedelta

class TweetRateLimiter:
    def __init__(self, daily_limit: int = 2400):
        self.daily_limit = daily_limit
        self.window = timedelta(days=1)
        self.user_posts: dict[str, deque[datetime]] = {}

    def can_post(self, user_id: str, now: datetime) -> bool:
        posts = self.user_posts.setdefault(user_id, deque())
        cutoff = now - self.window
        while posts and posts[0] < cutoff:
            posts.popleft()
        if len(posts) >= self.daily_limit:
            return False
        posts.append(now)
        return True

Tweet write
   -> Kafka (tweets topic, key=user_id)
   -> Heron stream job:
        if author.follower_count < 1M (normal user):
            FANOUT-ON-WRITE: push tweet_id to inbox of every follower
            (in Manhattan / Redis, sharded by follower_id)
        else (celebrity):
            FANOUT-ON-READ: tweet stays in author's outbox only
            on follower's timeline request, fetch celebrity outboxes
            (limited set, can be cached)

Timeline request
   -> Service merges:
       (a) follower's inbox (fanout-on-write portion)
       (b) celebrity outboxes for follows >1M (fanout-on-read portion)
   -> ML ranker scores combined candidate set
   -> top-K returned

Failure modes:
1. Heron worker crash: Kafka redelivers, idempotent inbox writes.
2. Celebrity inbox-fanout drift: rare; daily reconciliation job.
3. Cache eviction during traffic spike: degrade to direct Manhattan
   read with higher latency, no functional break.