Interview for Machine Learning

Tag: System

MSD0036 Domain Fine-Tuning Pipeline
Design a fine-tuning pipeline that adapts a foundation model to a company’s domain, including data curation, training strategy, and evaluation. A large enterprise wants to take a general-purpose LLM (e.g., Llama 3 or Mistral) and specialize it for its own support tickets, internal wikis, product manuals, and domain-specific jargon.
The pipeline must curate and clean training data from noisy heterogeneous sources, choose between full fine-tuning, LoRA/QLoRA, and instruction tuning, avoid catastrophic forgetting of general capabilities, and produce a reproducible evaluation that proves the adapted model is better on domain tasks without regressing on general benchmarks.
How would you design this pipeline? Cover the data curation and deduplication strategy, the fine-tuning approach and hyperparameter choices, the evaluation harness (domain-specific benchmarks plus general-capability guardrails), the infrastructure for reproducible runs, and how you decide when the adapted model is ready to replace the base model in production.

The Problem: the base model speaks fluent English but has never read a single one of your change requests, part numbers, or escalation runbooks, and roughly 900M tokens of company text are sitting unused.
Answer
The design is a curate, adapt, gate pipeline: a versioned data factory turns noisy internal sources into a deduplicated, PII-scrubbed corpus plus a hand-reviewed instruction set, a QLoRA adapter is trained on top of a pinned base checkpoint, and a dual-gate evaluation decides promotion. Two decisions dominate. First, adapt with parameter-efficient tuning on a small high-quality mix (domain corpus, instruction pairs, and 5 to 10 percent general replay data) rather than full fine-tuning on a raw dump, because a raw dump buys memorization and forgetting instead of capability. Second, make the promotion decision a two-sided gate: a required lift on a frozen domain benchmark and a bounded regression budget on general benchmarks, both measured by a versioned harness that any engineer can rerun from a run manifest.
(1) Data Factory Before Modeling: exact plus MinHash near-duplicate removal, PII and secret scrubbing, quality filtering, and access-control filtering, all emitting a content-hashed dataset version.
(2) Two-Stage Adaptation: short continued pretraining on the cleaned domain corpus to absorb jargon, then instruction tuning on curated ticket-resolution and doc-QA pairs to restore and shape behavior.
(3) QLoRA as the Default: 4-bit frozen base with rank-16 adapters on attention and MLP projections, so a run costs a single GPU-hour class of compute and the base weights stay bit-identical.
(4) Replay Against Forgetting: mix general instruction data into the SFT set and gate on general benchmarks, so domain gains are never paid for with instruction-following collapse.
(5) Reproducible Runs: pinned base digest, dataset hash, seed, container image, and config in one manifest; every eval number traces back to one manifest.
(6) Adapter Serving with a Kill Switch: serve base plus hot-swappable adapter behind a registry, roll out shadow → 5 percent canary → full, and roll back by unloading the adapter.

Figure 1: The loop: curated and versioned data feeds a cheap adapter run, the harness decides promotion, and production traffic becomes next month’s training data.
Clarify Before Designing:
(1) Target Tasks: which two or three tasks must improve (ticket triage, draft replies, doc QA), and who signs off on the frozen eval set for them?
(2) Corpus Reality: how many tokens survive after access-control and license filtering, and how much of the ticket volume is template boilerplate?
(3) Knowledge vs Behavior: is the gap stale facts (retrieval work) or unfamiliar jargon, format, and tone (fine-tuning work)?
(4) Regression Budget: how many points of MMLU, IFEval, or safety-refusal rate is the business willing to lose for a domain win?
(5) Serving Constraints: self-hosted weights or a closed API, one adapter for everyone or per-business-unit adapters, and what is the latency budget?
(6) Compliance: can customer PII enter training weights at all, and what is the deletion or right-to-forget story once it does?
Login to view more content
September 3, 2026
MSD0035 AI Agent Safeguards
Design the safeguards layer for an AI agent that can take real-world actions on a user’s behalf: booking flights, making purchases, sending messages, or modifying account settings, as envisioned for ChatGPT’s Operator, Anthropic’s computer use, and autonomous agent frameworks. Unlike a chatbot that only produces text, this agent can spend money and send irrevocable messages, so a prompt injection, a misread page, or a goal misinterpretation can cause real harm.
The safeguard system must approve or block each action before execution, verify that the action matches the user’s actual intent, defend against prompt injection from web content the agent reads, and provide a clear audit trail and undo path when something goes wrong.
How would you design this system? Cover the action classification and risk tiers, the intent verification mechanism (confirmation, biometric, second factor for high-risk), the prompt-injection defense for tool-returned content, the human-in-the-loop gate for irreversible actions, and how you evaluate safety without waiting for a real incident.

The Problem: the agent’s inputs include text written by strangers, and its outputs include charges, messages, and settings changes that cannot be taken back. Every action needs a gate before it reaches the world.
Answer
The design is an action firewall that sits between the agent’s planner and every tool that touches the world. Nothing the model “decides” executes directly. Each proposed action is emitted as a typed, structured call, assigned a risk tier by a cheap classifier plus deterministic rules, then evaluated by a policy engine against the capabilities and budget the user actually granted for this session. Two decisions carry most of the safety weight. First, untrusted tool output is quarantined data, never instructions, so page text cannot mint new goals or new capabilities. Second, irreversible actions pass through a human confirmation gate with step-up authentication, and everything else is made cheap to undo through logging, reversible-by-design tool wrappers, and an undo queue.
(1) Typed Action Schema: the agent cannot call raw HTTP or type free-form shell text; it emits validated calls such as purchase(merchant, amount, currency) so risk is computable from arguments, not guessed from prose.
(2) Four Risk Tiers: T0 read-only, T1 reversible writes, T2 spending or external messages, T3 irreversible or identity-changing (password, 2FA, transfers, mass sends); tier fixes the required evidence of intent.
(3) Deterministic Policy Engine: per-session capability grants, allowlists, per-action and per-session spend caps, rate limits, and recipient novelty checks, all enforced outside the model.
(4) Injection Containment: a planner that never sees raw untrusted text plus a quarantined extractor that returns typed values only, with data-flow provenance so tainted values cannot authorize a T2 or T3 action.
(5) Intent Verification Ladder: silent execution for T0/T1, in-context confirmation showing the exact parsed arguments for T2, and passkey or biometric step-up for T3, always rendered from the structured call rather than from model prose.
(6) Audit and Undo: append-only log of prompt, evidence, tier, policy decision, and result; reversible wrappers (hold instead of charge, delayed send, snapshot before settings change) give a real rollback path.

Figure 1: The action firewall: classify → check policy → decide, with untrusted tool output entering as quarantined data and irreversible actions detouring through human confirmation before the sandboxed executor runs.
Clarify Before Designing:
(1) Action Surface: which tools exist on day one (browser clicks, payment API, email, account settings), and can we wrap them in reversible primitives such as authorization holds and delayed sends?
(2) Autonomy Contract: does the user pre-authorize a budget and scope per session, or must every spend be confirmed live? This single answer moves confirmation volume by an order of magnitude.
(3) Harm Asymmetry: what is the cost of a wrongly executed purchase versus a wrongly blocked one, and who eats the loss (user, merchant, or platform)?
(4) Latency and Volume: how many actions per day, and how much of the per-action budget may the safeguard layer consume before the agent feels sluggish?
(5) Identity and Compliance: is a passkey or biometric available on the device, and do payments, health, or messaging rules mandate explicit consent records?
(6) Adversary Model: are we defending against opportunistic web-page injections only, or against attackers who will craft pages specifically targeting our agent?
Login to view more content
August 22, 2026
MSD0034 RAG Customer Support Assistant
Design a RAG-based customer support assistant for a company like Stripe, Shopify, or a large telecom. The assistant is grounded in the company’s own knowledge base: help articles, product documentation, past resolved tickets, and policy pages. It must answer accurately, cite its sources, refuse to fabricate when the answer is not in the knowledge base, escalate to a human when the question is high-stakes (billing, account access, legal) or when its confidence is low, and stay current as the knowledge base is edited every day.
The cost structure is asymmetric: a hallucinated refund policy or a confident wrong answer about account access is far worse than admitting ignorance and handing the conversation to an agent. A wrong answer can create a chargeback, a regulatory complaint, or a churned account, while an escalation costs a few dollars of agent time.
How would you design this system? Cover the retrieval architecture (chunking, embedding, hybrid search, re-ranking), the generation and grounding strategy, the confidence and safety gating that decides when to answer versus escalate, the human handoff path with full context, and how you evaluate answer quality and hallucination rate.

The Problem: the answer either exists somewhere in a knowledge base that changes daily, or it does not exist at all, and the system has to tell those two cases apart before it opens its mouth.
Answer
The design is a retrieve, re-rank, generate pipeline wrapped in an explicit answer policy. Hybrid retrieval over structure-aware chunks pulls 50 candidates, a cross-encoder re-ranker trims them to 8, and the generator may only assert what a cited passage actually says. Two decisions carry the answer. First, abstention and escalation are first-class outputs: a risk classifier runs before generation, a retrieval-confidence threshold guards coverage, and a per-claim groundedness verifier checks the draft, so the default under doubt is a warm human handoff rather than a fluent guess. Second, the index is versioned and refreshed nightly by delta, with product, plan, and region metadata as hard filters, because most support “hallucinations” are really retrieval of the wrong but plausible policy version.
(1) Structure-Aware Chunking: split on headings at roughly 400 tokens with 15% overlap, keep tables and code blocks intact, and prepend the article title and breadcrumb so an isolated chunk is self-describing.
(2) Rich Chunk Metadata: product, plan, region, locale, source type, version, and effective date travel with every chunk and act as hard filters before scoring.
(3) Hybrid Retrieval Plus Re-ranking: BM25 catches error codes, SKUs, and API field names; dense embeddings catch paraphrase; fusion gives 50 candidates and a cross-encoder picks the top 8.
(4) Citation-Constrained Generation: numbered passages in the prompt, sentence-level citation ids in a structured output, low temperature, and an explicit insufficient_context abstain option.
(5) Three-Gate Answer Policy: risk classifier, retrieval-confidence threshold, and claim-level groundedness check; any gate that fires routes to a human instead of shipping the text.
(6) Warm Handoff and Content Loop: the agent inherits the transcript, the exact passages used, the extracted entities, and a draft reply; the agent’s final answer becomes an eval label and, when the gap is real, a new article.

Figure 1: One pass per question: rewrite and risk-classify, retrieve and re-rank, generate only from the surviving passages, then let the gate decide whether the customer sees the text or an agent does.
Clarify Before Designing:
(1) Volume and Channels: how many chat sessions per day, how many turns each, which languages, and is the user authenticated when they ask?
(2) Quality Bar and Deflection Target: what accuracy floor must hold for answers we do give, and who signs off on the refusal rate that buys it?
(3) Knowledge Base State: how much content is written for customers versus internal agents, are policies versioned by region and plan, and are ticket resolutions quality-reviewed?
(4) Account Access: may the assistant read account state (plan, invoices, outage status) through tools, or only public docs? This changes both retrieval and the risk gate.
(5) Latency and Cost Budget: target first token, target full answer, and the per-conversation cost ceiling relative to an agent contact.
(6) High-Stakes Definition: which intents are legally or financially sensitive (billing disputes, cancellation, identity, data deletion) and what retention rules apply to logged conversations?
Login to view more content
August 22, 2026
MSD0031 Autonomous Sensor Fusion
Design the sensor fusion system for an autonomous vehicle that combines camera, LiDAR, and radar into a unified perception of the world, as used by Waymo, Cruise, and Tesla’s Full Self-Driving stack. Each sensor has complementary strengths (camera: dense semantic and color detail; LiDAR: accurate depth and 3D structure; radar: velocity and all-weather range) and weaknesses (camera: depth ambiguity and weather sensitivity; LiDAR: sparse, expensive, poor in fog and rain; radar: low angular resolution and clutter). The fused perception must feed detection, tracking, and prediction at 10-20 Hz with hard latency bounds, and it must degrade gracefully when a sensor fails or is occluded rather than producing a confident wrong answer.
How would you design this system? Cover the per-sensor processing, the fusion architecture (early, late, or mid-level BEV fusion), the time synchronization and calibration, how you handle sensor dropout and degradation, and how you evaluate fusion quality against single-sensor baselines.

The Problem: three sensors look at the same road spray and return three different stories, and the vehicle still has to emit one confident answer every 50 to 100 ms.
Answer
Build a mid-level fusion stack over a shared bird’s-eye-view (BEV) grid. Each sensor keeps its own encoder, every feature is warped into one metric grid at one reference timestamp, and a lightweight transformer fuses them before the detection, occupancy, and tracking heads. Mid-level fusion is the pivotal choice: early raw fusion is brittle under calibration drift and blocks per-sensor pretraining, while late fusion discards exactly the cross-modal evidence that resolves camera depth ambiguity and rescues weak single-sensor detections. Two further decisions carry the safety argument. Train with modality dropout so the fused model remains usable when a sensor disappears, and keep a small non-learned geometric path (radar Doppler tracks plus LiDAR clustering) that can hold the vehicle to a safe stop if the learned stack degrades.
(1) Shared BEV Representation: project all modalities into a common ego-centric grid (roughly 100 m by 100 m at 0.4 m resolution) so fusion is a geometric alignment problem rather than an ad hoc feature concatenation.
(2) Per-Sensor Encoders: depth-distribution lift for images, sparse voxel or pillar encoder for LiDAR, and a point encoder that keeps radar range-rate; each branch trains and validates independently.
(3) Soft-Association Fusion: cross-attention between object queries and per-sensor features instead of hard pixel-to-point matching, which keeps accuracy when extrinsics drift by a fraction of a degree.
(4) Hardware Sync and Online Calibration: PTP-locked triggers, per-point timestamps, and ego-motion warping to one reference time, with reprojection residuals monitored continuously as a calibration health signal.
(5) Fallback Ladder: a health monitor scores each sensor and steps the system from full fusion, to degraded fusion with widened safety margins, to the geometric path plus an ODD exit or minimal risk maneuver.
(6) Ablation-Gated Evaluation: the release rule is that fused perception is never worse than the best single-sensor baseline on any evaluation slice, including night, rain, and forced-dropout slices.

Figure 1: One learned path through a shared BEV grid, plus a health monitor and a non-learned geometric path that stays alive when the learned path should not be trusted.
Clarify Before Designing:
(1) Sensor Suite and Cost Ceiling: how many cameras, what LiDAR (spinning 64-beam, solid state, or none), how many radars, and is 4D imaging radar available?
(2) Operational Design Domain: geofenced urban robotaxi with HD maps, or consumer highway assist without maps; does the ODD include heavy rain, snow, and night?
(3) Compute and Latency Budget: on-vehicle TOPS, power and thermal envelope, target rate (10 Hz or 20 Hz), and the p99 rather than mean latency bound.
(4) Failure Policy: on sensor loss, does the vehicle degrade speed, exit the ODD, hand back to a driver, or execute a minimal risk maneuver; who signs off on that decision?
(5) Label Supply: how much human-labeled 3D data exists, and can an offboard auto-labeling pipeline produce pseudo ground truth from full logs?
(6) Redundancy Requirement: must perception survive the loss of any single sensor, or any single compute unit, and is a second independent stack mandated?
Login to view more content
August 22, 2026
MSD0030 Google Maps Traffic Prediction
Design the traffic prediction system behind Google Maps’ ETA and route recommendations. The system must forecast road speeds and congestion on every segment of a global road network, minutes to hours ahead, so that routing can pick the fastest path and display accurate arrival times.
It must fuse real-time probe data from millions of phones and fleet vehicles, historical patterns, live incidents (accidents, construction, events), and weather. It also has to respect the spatial and temporal dependencies that make a jam on one freeway cascade onto adjacent arterials minutes later.
How would you design this system? Cover the data ingestion and map-matching pipeline, the spatio-temporal forecasting model, how real-time and historical signals are combined, how incidents and events are incorporated, and how the ETA is computed and kept honest as conditions change.

The Problem: a jam does not stay on one road and probe coverage is uneven, yet the product must commit to a single arrival time for every trip, everywhere, right now.
Answer
The design is a streaming pipeline that turns anonymized probe traces into per-segment speed observations, then a spatio-temporal graph neural network over the road network that predicts a multi-horizon quantile speed distribution for every segment. Predictions are precomputed into a distributed speed store, and the routing engine reads them as time-dependent edge weights, so a route query never waits on a model call. Two decisions are pivotal. First, the model operates on supersegments (chains of consecutive segments that share traffic) rather than isolated edges, which fixes probe sparsity and lets message passing carry a jam upstream and onto parallel arterials. Second, the real-time signal is a residual on top of a historical speed profile, not a replacement for it, so accuracy degrades gracefully as probe density drops and as the horizon grows. Google’s published version of this system reports ETA accuracy improvements of up to 50% in several metros after moving from segment-local models to a GNN over supersegments.
(1) Map Matching And Aggregation: HMM map matching snaps noisy GPS traces to road segments, and traversals are aggregated into short time bins per segment and supersegment so no individual trip is recoverable.
(2) Historical Prior Plus Live Residual: a per-segment, per-time-of-week speed profile is the baseline; the live signal predicts the deviation from it, which is what makes sparse-probe roads survivable.
(3) Spatio-Temporal GNN: nodes are segments with live, historical, and static features; two to three rounds of message passing propagate congestion along the network, and multi-horizon heads emit speeds at 2, 10, 30, and 60 minutes.
(4) Incidents As Features Plus A Short-Horizon Override: accidents, closures, events, and weather enter as node and context features, with a corroborated override path that patches the next few minutes faster than the model can retrain.
(5) Precompute And Serve: predictions land in a segment-by-horizon key-value store read in single-digit milliseconds; routing runs time-dependent search so long trips use future speeds, not present ones.
(6) Quantile ETA And Mid-Trip Correction: the product shows a calibrated interval rather than a point estimate, and the ETA is re-estimated during the trip as the vehicle’s own trace becomes evidence.

Figure 1: The serving path: probes become segment observations, the model writes future speeds ahead of time, and the router only reads. Realized travel times close the loop as training labels.
Clarify Before Designing:
(1) Coverage And Probe Density: which markets, and what fraction of segments see enough probes in a five-minute window to support a live estimate at all?
(2) Horizon And Refresh: do we only serve depart-now ETAs, or also scheduled departures hours ahead, and how often must each segment be refreshed?
(3) Latency Budget: what is the end-to-end budget for a route query, and is the prediction allowed to run inline or must it be precomputed?
(4) Error Asymmetry: is arriving late worse than arriving early, and by how much, since that decides which quantile we display?
(5) Privacy Constraints: what aggregation thresholds and retention limits apply to location traces, and do they differ by region?
(6) Routing Ownership: do we control the routing engine (so we can change edge weights and spread traffic) or only publish speeds to a consumer?
Login to view more content
August 22, 2026
MSD0028 Deepfake AIGC Detection
Design a deepfake and AI-generated content detection system for a video platform like YouTube or TikTok. Generative video models (Sora, Runway, Kling) and voice cloning tools have made convincing synthetic media cheap, and the platform must label or remove media that misleads viewers, impersonates real people, or spreads disinformation, while leaving legitimate creative and artistic uses of the same tools untouched.
The detector has to keep up with generators that improve every few months, survive uploads that were re-encoded and re-compressed until most forensic cues are gone, and avoid flagging real content that merely looks synthetic (high-quality CGI, heavily graded or filtered footage, screen recordings).
How would you design this system? Cover the detection signals (frequency artifacts, temporal inconsistencies, biometric cues, provenance metadata like C2PA), the model architecture for image, audio, and video, how you handle the adversarial arms race against improving generators, the human-review and appeals path, and how you measure detection rate and false-positive rate.

The Problem: three clips reach the same viewer, one filmed, one generated, one hand-made CGI. Only the misleading one should be acted on, and the pixels have already been re-encoded to 480p before you ever see them.
Answer
The design is a provenance-first, harm-tiered cascade, not a binary “fake or real” classifier. Every upload is first checked for signed C2PA Content Credentials and invisible watermarks, then a cheap multimodal triage model scores sampled frames plus the audio track, and only the riskiest few percent enter a heavy visual, audio, and temporal ensemble. The two pivotal decisions are that the system outputs a calibrated risk score plus a harm class (impersonation of a real person, medical or election claim, synthetic-but-harmless art) which drives graduated enforcement rather than blanket removal, and that the whole training and evaluation loop is built around the arms race: continuous harvesting of every new generator family, red teaming, and a release gate measured on unseen-generator holdouts and in-the-wild data instead of academic benchmarks.
(1) Provenance First, Detection Second: validate C2PA manifests and watermark decoders before any classifier runs; a valid signature is strong positive evidence, but a missing one proves nothing because metadata is stripped by every transcode.
(2) Cheap Triage, Expensive Ensemble: a small model scores 100% of uploads, the deep ensemble runs on roughly 2%, keeping compute inside budget while preserving recall on the tail that matters.
(3) Multimodal Signal Stack: spatial and frequency artifacts, temporal cues (landmark jitter, blink rate, rPPG pulse, lip-sync offset), and audio cues (vocoder phase artifacts) fused late, so no single fragile signal owns the decision.
(4) Compression-Robust Training: synthetic training data is pushed through the platform’s own transcode ladder, because a detector trained on pristine generator output collapses on real uploads.
(5) Harm-Tiered Enforcement: disclosure label, then demotion, then human review, then removal; full automation only for narrow high-confidence classes such as a cloned voice of a verified public figure.
(6) Arms-Race Loop: a red team plus a generation farm produce fresh fakes from every new model release, feeding a 2 to 4 week retrain cadence with an emergency path.

Figure 1: One cheap pass over everything, one expensive pass over the risky tail, and a review loop whose overturns become next week’s training data.
Clarify Before Designing:
(1) Policy Scope: are we labeling all synthetic media, or only synthetic media that misleads (impersonation, fabricated events, health and election claims)? The answer changes the target variable itself.
(2) Scale and Latency: uploads per day and peak factor, and must a verdict land before publish or within minutes after, before the video can go viral?
(3) Provenance Coverage: what fraction of uploads arrive with valid C2PA credentials or a decodable watermark today, and which camera and tool partners are signed up?
(4) Ground Truth: do we have licensed API access to the major generators for in-house fake production, and how many human-confirmed in-the-wild labels per week does review produce?
(5) Cost Asymmetry: what does a missed political deepfake cost versus a wrongly demoted VFX creator, and how large is the review headcount that sets our false-positive budget?
(6) Jurisdiction: which transparency rules apply (EU AI Act disclosure duties, election-period rules), and do they force labels regardless of harm?
Login to view more content
August 22, 2026
MSD0027 Fake Account Bot Detection
Design a system to detect fake accounts and bots on a social network like Facebook, Twitter/X, or Instagram. Fake accounts power spam, coordinated inauthentic behavior, follower farming, and political manipulation, and their operators evolve constantly: they buy aged accounts with real history, use LLMs to write bios and mimic human posting cadence, and route traffic through residential proxies so IP reputation says nothing.
The system must score accounts at creation time and keep re-evaluating them as behavior develops, while holding the false-positive rate low enough that real users are not wrongly suspended, and routing the highest-risk cases to human investigators.
How would you design this system? Cover the account-level feature signals (registration, graph, content, behavior), the detection model architecture, how you handle adversarial evolution, the human-in-the-loop review queue, and how you measure precision and recall when ground truth is scarce.

The Problem: millions of new accounts a day, a farm that can forge everything cheap to forge, and one real person sitting in the same batch. Decide who to trust without suspending them.
Answer
Treat this as two coupled problems running at two speeds. At signup, a low-latency gradient-boosted scorer over registration, device, and infrastructure features decides whether the account starts with friction, and a continuous re-scoring layer then re-judges every live account as its graph and behavior accumulate. That second layer is where the real detections happen: a relational GNN over the follow and friend graph plus unsupervised synchrony clustering (the SynchroTrap idea of co-clustering accounts that act on the same targets in the same time buckets) catches campaigns whose individual members each look perfectly human. Scores never map directly to bans. They map to a graduated enforcement ladder (allow, challenge, limit reach, human review, disable) so the expensive mistake, suspending a real user, requires either overwhelming cluster-level evidence or an investigator.
(1) Two-Speed Detection: a synchronous per-signup scorer under a 200 ms budget, plus a daily (and event-triggered) batch pass over every active account, since most fake accounts only reveal themselves after they start acting.
(2) Cluster-Level Evidence: the decision unit is the account and the campaign; graph plus action-synchrony clustering finds coordinated groups without needing labels for the new tactic.
(3) Cost-To-Forge Feature Weighting: prefer signals an operator must pay real money or real time to fake (device integrity, aged organic graph, human-only interaction telemetry) over signals a proxy or an LLM erases for free.
(4) Graduated Enforcement: friction first (phone or CAPTCHA challenge, reach limiting), auto-disable only in the top band with corroboration, always with an appeal path that feeds labels back.
(5) Label Machinery: reviewer verdicts, appeal reversals, honeypot and test-buy cohorts, and a random-audit stream, because production has no clean ground truth and the flagged sample alone teaches the model its own blind spots.

Figure 1: Two speeds, one policy: a synchronous scorer at signup, a daily graph and synchrony pass over every live account, and a ladder whose top band ends at a human, not a model.
Clarify Before Designing:
(1) Scale And Prevalence: how many signups per day, how many active accounts to re-score, and what does the current prevalence estimate say (a few percent of MAU)?
(2) False-Positive Budget: how many wrongly restricted real users per day is the product willing to accept, and is auto-suspension allowed at all under the platform’s appeal and regulatory commitments?
(3) Enforcement Surface: can we impose signup friction (phone verification, CAPTCHA), or does growth own that decision and force us to detect after the fact?
(4) Review Capacity: how many investigator-hours per day exist, in which languages and regions, and what is the target queue latency for high-harm cases?
(5) Threat Priority: which abuse type pays first, follower farms, spam, or coordinated political campaigns, since they differ in scale, sophistication, and tolerance for delay?
(6) Signal Access: what device, telemetry, and payment signals are legally usable per region, and how long may we retain them?
Login to view more content
August 22, 2026
MSD0026 Multimodal Text-to-Image-Video Search
Design a multimodal search system where a user types a text query and the system returns relevant items from a catalog of billions of images and videos, like Pinterest Visual Search, Google Lens, or TikTok’s visual search. The query may describe objects, scenes, styles, or actions that never appear as literal text anywhere in the catalog (for example “golden retriever catching a frisbee at sunset”), so the system must bridge the modality gap between language and visual content.
Results must return in under 200 ms, the index must stay fresh as new content is uploaded continuously, and the system must handle queries that mix text with an uploaded reference image.
How would you design this system? Cover the shared embedding space and training strategy, the approximate nearest neighbor index at billion scale, the freshness and update strategy, the hybrid text-plus-image query path, and how you evaluate retrieval quality.

The Problem: the words the user types appear nowhere in the catalog, yet the right pixels have to surface out of billions of items inside 200 ms, while new uploads keep arriving.
Answer
The design is a two-tower (dual encoder) retrieval system over one shared embedding space, served as a two-stage retrieve-and-rerank path. A contrastive image-text model of the SigLIP/CLIP family, fine-tuned on in-domain query-engagement pairs, maps text, images, and video clips into the same 512-d space, so “golden retriever catching a frisbee at sunset” becomes a vector whose neighbors are the right pixels even though no matching caption exists. Two decisions carry most of the weight. First, the item side is multi-vector for video (a handful of clip embeddings rather than one averaged vector), because averaging a 60-second video destroys the action the query describes. Second, freshness is solved with a tiered index (hot in-memory buffer, hourly delta shards, nightly rebuilt base) instead of mutating one billion-scale graph in place. The hybrid text-plus-image query runs through a learned combiner that emits a single vector, so the retrieval and ranking path downstream never changes.
(1) Shared Embedding Space: one contrastively trained space for query text, images, and video clips, so retrieval is pure vector search rather than text matching against noisy alt-text.
(2) In-Domain Fine-Tuning: start from a public image-text checkpoint, then fine-tune on click and save pairs from real query logs, with hard negatives mined from the same logs.
(3) Multi-Vector Video Items: sample frames or short clips per video and index several vectors per item, collapsing to one item at merge time; single pooled vectors lose action and scene changes.
(4) Sharded IVF-PQ With Rerank: compress to 64-byte codes so 5 billion vectors fit in roughly 320 GB of RAM across shards, retrieve about 10k candidates, then rescore with full-precision vectors plus business features.
(5) Tiered Freshness: a hot buffer makes new uploads searchable in about a minute, hourly deltas absorb the day, and a nightly base rebuild keeps the big index compact; deletes ride a tombstone list.
(6) Learned Query Combiner: text-plus-image queries are fused into one vector by a small trained combiner, keeping the index path and latency budget identical to text-only search.

Figure 1: One embedding space, two write paths: queries are encoded online, catalog items are encoded on ingest, and the tiered index is the only place they meet.
Clarify Before Designing:
(1) Scale and traffic: how many items (billions of images, or billions of items with heavy video), what peak QPS, and how many uploads per hour?
(2) Latency budget split: is the 200 ms end-to-end including network and thumbnail fetch, or only the retrieval service?
(3) Freshness requirement: must a new upload be findable in seconds (live events, breaking trends) or is an hour acceptable for most content?
(4) Query mix: what fraction is head navigational text, tail descriptive text, action queries over video, and hybrid image-plus-text?
(5) Quality bar and cost asymmetry: is the product optimizing engagement, commercial conversion, or safety-first precision, and what is an acceptable cost per 1k queries?
(6) Available supervision: do we have query-click logs and human relevance ratings, or only alt-text and hashtags?
Login to view more content
August 22, 2026
MSD0025 Dating App Match Recommendation
Design the match recommendation system for a dating app like Tinder or Hinge. When a user opens the app, the system must surface a stack of candidate profiles that the user is likely to like and who are also likely to like them back, because a one-sided like is wasted attention.
The system must handle the severe imbalance in activity and desirability (a small fraction of profiles receives most of the likes), respect hard constraints (distance, age, stated preferences), introduce enough novelty to avoid the filter bubble where users see the same type over and over, and balance short-term engagement against long-term retention: a user who gets no matches churns, but so does one who is overwhelmed.
How would you design this system? Cover the candidate generation and ranking funnel, the two-sided reciprocity model, how you handle the popularity imbalance, the exploration strategy, and the online and offline metrics.

The Problem: every card in the stack spends a user’s finite attention, and a like that is never returned buys nothing for either side. The design question is which profiles deserve those thirty slots.
Answer
The design is a four-stage funnel: hard filters cut the global user base to an eligible geo and preference pool, an embedding retrieval stage pulls roughly a thousand candidates, a reciprocal ranker scores each pair by the probability of a mutual like, and a slate builder assembles the actual stack under diversity, exposure, and exploration quotas. Two decisions carry the answer. First, the ranking objective is expected mutual match, $p_{uv} \cdot p_{vu}$ , not one-sided like probability, because a one-sided like is wasted attention on both sides of the market. Second, the system is treated as a two-sided marketplace with a scarce resource (each user’s daily attention and inbox capacity), so popular profiles get exposure caps and under-exposed profiles get guaranteed slots instead of being buried by a pure desirability ranking.
(1) Hard Filters First: distance, age band, gender preference, blocks, and recent-activity recency are non-negotiable constraints applied before any model, shrinking millions of profiles to tens of thousands.
(2) Two-Tower Retrieval: geo-sharded ANN lookup over profile embeddings plus complementary sources (mutual-graph, recently-active, likes-you inbox) returns about a thousand candidates in tens of milliseconds.
(3) Reciprocal Ranker: a shared-encoder model with a forward head and a reverse head predicts both directions and ranks by the calibrated product, so a wildly popular profile who will never swipe back scores below a plausible mutual.
(4) Exposure Allocation: per-profile impression caps, inbox-load damping, and demand-aware discounting spread likes across the tail instead of concentrating them on the top decile.
(5) Slate With Explore Slots: the stack is built as a slate, not a top-k list, with quotas for diverse archetypes, uncertain pairs, and new users, which is also how the system escapes the filter bubble.
(6) Retention-Aligned Objective: the training label mix and slate quotas are tuned against day-7 and day-28 retention and conversation rate, not swipe volume.

Figure 1: The funnel: filters remove the impossible, retrieval proposes, the reciprocal ranker scores both directions, and the slate builder decides what the user actually sees.
Clarify Before Designing:
(1) Scale And Density: how many daily active users, and how dense is a typical market (a dense city with 50k eligible profiles behaves nothing like a rural area with 300)?
(2) Product Grammar: is it a fast swipe stack (Tinder style, roughly 100 cards per session) or a small curated set (Hinge style, a handful per day)? The slate size changes the whole exploration budget.
(3) Success Definition: is the north star matches, conversations that reach a reply, dates, or subscription retention? Optimizing matches alone rewards low-quality likes.
(4) Latency And Precompute: can we precompute a stack asynchronously between sessions, or must every card be scored at request time?
(5) Fairness And Policy Constraints: which attributes may the ranker use, what exposure floor do we owe new or low-activity users, and are paid boosts part of the allocation?
(6) Cost Asymmetry: which is worse for retention, a session with zero appealing profiles or an inbox the user cannot keep up with?
Login to view more content
August 22, 2026
MSD0024 LinkedIn Job Recommendation
Design the job recommendation system for LinkedIn Jobs. When a member visits the Jobs page or receives a job alert email, the system must rank open positions from a catalog of millions by how likely the member is to apply and, ultimately, to get hired.
The ranking must respect hard constraints (location, work authorization, years of experience, salary expectations), handle the cold-start problem for new members and newly posted jobs, and avoid recommending jobs the member has already seen or dismissed. LinkedIn’s two-sided marketplace means the system also affects employers, who pay for applications and churn if candidate quality is poor.
How would you design this system? Cover the candidate generation and ranking funnel, the feature set (member profile, job requirements, interaction history), how you handle cold start, the online and offline metrics, and how you balance member and employer value.

The Problem: one member with hard eligibility constraints, millions of open postings, and a recruiter on the other side who pays for applications and leaves if the applicants are unqualified.
Answer
The design is a standard four-stage recommender funnel wrapped around a two-sided objective: hard-constraint filtering, multi-source candidate generation down to roughly a thousand jobs, a multi-task ranker, then a policy re-rank layer that enforces marketplace and compliance rules. The pivotal design decision is what the ranker optimizes. Optimizing apply probability alone maximizes member clicks and floods a few popular postings with unqualified applications, so the score blends P(apply), P(recruiter treats the application as qualified), and a delayed P(hire) head, which is the objective LinkedIn’s own confirmed-hires framing points at. The second pivotal decision is to treat eligibility as a pre-filter in the retrieval index rather than a learned soft feature, because a job the member cannot legally take is a wasted slot for both sides.
(1) Hard Constraints As Pre-Filters: location radius or remote flag, work authorization, seniority window, and compensation band are boolean index filters applied before scoring, never soft features the ranker can trade away.
(2) Multi-Source Candidate Generation: a two-tower ANN over job embeddings, plus saved searches and alerts, plus graph sources (“members with your title applied here”) and company affinity, unioned to about 1000 candidates.
(3) Multi-Task Ranker: one shared-bottom or MMoE model with heads for apply, qualified-application, and confirmed hire; the serving score is a weighted combination whose weights are the marketplace policy knob.
(4) Cold Start By Content Plus Exploration: both towers are content-based (title, extracted skills, seniority, company), so new members and postings get vectors immediately, and new postings receive a small guaranteed impression budget.
(5) Policy Re-Rank Layer: deduplicate seen and dismissed jobs, cap applicant load per posting, spread traffic beyond the head of the catalog, and apply fairness-aware re-ranking.
(6) Two Serving Paths: the Jobs page runs the funnel online under a ~200 ms budget; alert emails run the same models in a nightly batch with frequency caps and a higher score threshold.

Figure 1: The funnel narrows 20M postings to 25 shown jobs, and the loop back through label joining is what turns applies, recruiter replies, and hires into tomorrow’s model.
Clarify Before Designing:
(1) Surface And Budget: Jobs page (online, latency-bound) or alert email (batch, frequency-bound), and what end-to-end latency budget does the page allow?
(2) Label Availability: do we have recruiter-side signals (application viewed, candidate contacted) and confirmed hires, or only member-side clicks and applies?
(3) Catalog Scale And Freshness: how many open postings, what is the median posting lifetime, and how quickly must closed jobs disappear?
(4) Monetization: are promoted or sponsored jobs blended into the same slots, and is there a separate auction and pacing layer?
(5) Compliance: which fairness and audit requirements apply (employment discrimination law, automated hiring-tool bias audits), and which member attributes are forbidden as features?
(6) Employer Capacity: do postings expose applicant caps or response-time signals we can use to protect candidate quality?
Login to view more content
August 22, 2026