Interview for Machine Learning

Category: ML

MSD0014 Music Identification
Design a music identification system like Shazam: a user holds up a phone in a noisy cafe, records a 10-second snippet, and the app names the song within a second. Apple’s ShazamKit exposes the same matching to third-party apps, Google’s “Now Playing” identifies music continuously and offline on Pixel phones, and SoundHound even accepts hummed melodies.
How would you design this system? Cover the audio fingerprint representation, the matching index, the latency and scale budget, and how the system behaves under noise, tempo changes, and remixes.

The Problem: ten seconds of music buried in cafe noise must pick one recording out of tens of millions, in under a second, and a wrong answer is worse than none. Design the system that hears the song through the crowd.
Answer
The design is audio fingerprinting with landmark hashes, the approach Shazam published in 2003 and which remains the backbone of production matchers: convert audio to a spectrogram, keep only the strongest time-frequency peaks (a constellation map that survives noise), hash pairs of peaks into compact tokens, and match a query by finding reference tracks whose hashes recur with one consistent time offset. The two pivotal decisions are to match a lossy peak representation instead of the waveform (robustness to noise, codec compression, and reverb) and to use an inverted index over hashes with time-coherence voting, which turns “find this song” into a few hundred lookups and a histogram test rather than a comparison against the whole catalog.
(1) Constellation Frontend: STFT spectrogram, keep local peaks only; noise must out-shout a peak to erase it, so ~20 peaks/second survive a cafe.
(2) Combinatorial Hashing: each anchor peak pairs with peaks in a short target zone; the hash packs both frequencies and the time gap, giving millions of distinct tokens.
(3) Inverted Index + Offset Voting: hash to list of (track, time) postings; matching query hashes vote on the offset, and a histogram spike declares the track.
(4) Asymmetric Confidence: naming the wrong song costs more than “no match found”, so an answer needs enough aligned votes plus a margin over the runner-up.
(5) Catalog Ingestion at Scale: new releases are fingerprinted as labels deliver them; the index is sharded by hash and held in RAM for sub-second queries.

Figure 1: Register and recognize lanes share one fingerprinting frontend; recognition is index lookups plus offset voting, never a comparison against raw audio.
Clarify Before Designing:
(1) Query Conditions: phone microphone in noise versus clean line-in audio, and how short a snippet must work (5 s? 15 s?)?
(2) Catalog Scope: tens of millions of commercial tracks, and do covers, remixes, and live versions count as the same song?
(3) Latency Budget: sub-second answer on a mobile connection, or is a few seconds acceptable?
(4) On-Device versus Cloud: must matching run offline with a regional catalog subset (the Now Playing design), or is the phone always online?
(5) Beyond Exact Recordings: is humming or cover identification in scope (a different, melody-based representation), or exact audio only?
Login to view more content
August 9, 2026
MSD0013 Machine Translation at Scale
Design a machine translation system for a product like Google Translate: text translation between 100+ language pairs, from high-resource pairs like English-Spanish down to low-resource languages with little parallel data, served interactively at web scale where users expect answers in a fraction of a second.
How would you design this system? Cover the model architecture, how you get training data for low-resource languages, the serving design, and how you measure translation quality.

The Problem: 100 languages mean roughly 10,000 directed pairs, most with almost no parallel data, and users still expect every pair to answer in milliseconds. Design one system that covers them all.
Answer
The design is a single massively multilingual encoder-decoder model trained on mined bitext plus back-translated monolingual text, with a target-language token steering generation (the NLLB recipe), behind a dual-track serving layer: a fast NMT track for short, latency-critical, terminology-sensitive text, and a fine-tuned translation LLM for long, nuanced, document-level input, which is exactly the split Google shipped in its Cloud Translation API (2024) and DeepL shipped as its next-gen model. The two pivotal decisions are the multilingual pooling (high-resource pairs teach low-resource ones through a shared representation, enabling zero-shot pairs) and the honest admission that one model family does not win everywhere: NMT still wins on latency, cost, and determinism, the LLM wins on context and fluency.
(1) One Multilingual Model: shared encoder-decoder with a target-language token; transfer from high-resource pairs gives zero-shot coverage of unseen pairs instead of 10,000 separate models.
(2) Data Flywheel: mine bitext from web crawls with multilingual sentence embeddings (LASER), then back-translate monolingual text to synthesize pairs for low-resource languages.
(3) Dual-Track Serving: NMT for short/interactive/terminology-critical requests, translation LLM for long-form and idiomatic text; a router picks per request.
(4) Quality Measurement: automatic metrics (COMET, BLEU) for regression, human MQM error marking for truth, A/B tests for product impact, all reported per language pair.
(5) Low-Resource Strategy: zero-shot transfer first, then few-example adaptive conditioning (Google’s Adaptive Translation shows as few as 5, and up to 30k, example pairs can steer output at inference time).

Figure 1: Dual-track serving over one multilingual training base: the router sends short, latency-critical text to NMT and long, context-heavy text to the translation LLM; terminology constraints apply on both tracks.
Clarify Before Designing:
(1) Scope: text only, or also speech and image translation; and which language pairs at launch?
(2) Latency Budget: interactive typing (sub-300 ms) versus document translation (seconds) imply different model families.
(3) Quality Bar: gist-level understanding for users, or publication-grade output with human post-editing?
(4) Terminology: must product names, legal terms, or brand glossaries be honored exactly?
(5) Customization: do enterprise customers need domain adaptation (medical, legal) with their own example pairs?
Login to view more content
August 9, 2026
MSD0012 Wake-Word Detection
Design a wake-word detection system (e.g., Amazon’s “Alexa” or Apple’s “Hey Siri”) that spots a trigger phrase in a continuous audio stream on a phone or smart speaker. It must listen always-on without draining the battery, wake almost instantly when the user speaks, and almost never fire by mistake.
How would you design this system? Cover the on-device architecture, the detection model, the training data, and how you measure accuracy.

The Problem: the detector discards nearly 100% of everything it hears, yet the one time it must fire, it has milliseconds and milliwatts to decide. Design the system that never sleeps but barely sips power.
Answer
The design is a two-stage cascade split across processors, the architecture Apple describes for its voice trigger system: a tiny streaming detector runs on the always-on low-power processor (DSP/AOP) tuned for high recall, and when it fires, a buffered audio segment goes to a larger, high-precision checker on the main application processor, optionally followed by speaker verification so only the enrolled user wakes the device. The two pivotal decisions are the cascade itself (the power budget makes one big always-on model impossible) and the operating point: false accepts are counted per unit of listening time (research reports them per hour of audio, products budget them per device-day) and they dominate the cost, so thresholds are tuned on an FA-versus-miss curve, not on accuracy.
(1) Cascade Across Processors: stage 1 is a tiny quantized DNN on the always-on processor (high recall, permissive); stage 2 is a larger checker on the application processor (high precision) fed from a ring buffer.
(2) Streaming Frontend: 16 kHz audio, 10 ms hops, log-mel filterbank frames, DNN posteriors over sub-word states, and a temporal integrator (HMM-style) that turns frame scores into one detection score.
(3) Asymmetric Objective: false accepts per device-day is the headline metric; a second-chance threshold band briefly raises sensitivity after a near-miss instead of paying FA cost all the time.
(4) Data Engine: real trigger recordings plus TTS-synthesized ones, augmented with noise, reverb, and far-field impulse responses, plus hard negatives (e.g., “seriously” for “Hey Siri”).
(5) Privacy by Construction: no audio leaves the device before activation; personalization (speaker ID) runs on-device after enrollment.

Figure 1: The cascade: a high-recall listener on the low-power chip gates a high-precision checker on the main chip; a ring buffer carries the candidate audio across so no trigger onset is lost.
Clarify Before Designing:
(1) Device Class: phone, watch, or smart speaker; battery and DSP availability decide stage sizes (a watch gets ~5% of a tiny compute budget).
(2) Trigger Spec: one phrase or several, fixed phrase or user-custom, and must it work hands-free in far-field rooms?
(3) False-Accept Budget: how many accidental activations per day are tolerable, and what do they cost (battery, privacy review, user trust)?
(4) Personalization: should only the enrolled owner’s voice trigger it, or anyone’s?
(5) Cloud Role: may a cloud verifier re-check activations, or is the device fully offline?
Login to view more content
August 9, 2026
MSD0011 LLM Evaluation Pipeline
Your team ships an LLM-powered product: a customer-support copilot built on a foundation model API, similar in spirit to how GitHub evaluates models for Copilot before each release. The model version, prompts, retrieval index, and tools all change every few weeks, and every change can silently alter answer quality.
Design an evaluation pipeline that runs before each release. What do you test, how do you score open-ended outputs, and how do you decide whether the new version is safe to ship?

The Problem: the system under test is stochastic, so a green build means nothing without a repeatable quality signal. Design the gate between “we changed the prompt” and “customers see it.”
Answer
The design is a CI pipeline for a stochastic system: a curated golden suite mined from real production failures, a grader ladder (deterministic checks where possible, an LLM judge calibrated against human labels where necessary, human audits on top), and a release gate that compares the candidate against the current version on quality metrics plus guardrails (safety, latency, cost). GitHub runs more than 4,000 such offline tests in CI before any Copilot model change; Anthropic’s guidance is to start with 20 to 50 tasks drawn from real failures and grow from there. The pipeline ends in a canary and A/B test, and production monitoring feeds new failures back into the suite, so the eval set is a living artifact.
(1) Cases From Reality: the golden suite is mined from production logs, bug reports, and support tickets, not invented at a desk; 20-50 real tasks beat 500 synthetic ones.
(2) Grader Ladder: deterministic checks first, an LLM judge (calibrated to human labels, pairwise where possible) second, human audits as the calibration anchor.
(3) Capability vs Regression Suites: capability evals start at a low pass rate and measure progress; regression evals sit near 100% and block backsliding; saturated capability cases graduate into the regression suite.
(4) Release Gate With Guardrails: the candidate must beat or tie the incumbent on quality with statistical significance, and must not breach guardrails on safety, latency, and cost.
(5) Flywheel: production failures are mined, de-identified, and converted into new eval cases every release cycle.

Figure 1: The pre-release path: golden suite, grader ladder, gate, then canary. The loop back from production logs is what keeps the suite honest as user behavior shifts.
Clarify Before Designing:
(1) Product Surface: single-turn answers, multi-turn chat, or an agent with tool calls; the grader design differs sharply.
(2) What Changes per Release: model version, prompts, retrieval data, tools; each change type needs its own regression coverage.
(3) Failure Cost: what is the worst plausible wrong answer (refund mistakes, medical tone, leaked PII) and who is accountable for it?
(4) Human Label Budget: how many expert labels per cycle can we afford, since they anchor the judge’s calibration?
(5) Release Cadence: weekly prompt tweaks and quarterly model upgrades imply different gate depths.
Login to view more content
August 9, 2026
MSD0010 Grammar Checking Writing Assistant

You are building a writing-assistance product like Grammarly. As a user types in a browser extension, email client, or mobile keyboard, the system underlines mistakes and offers corrections in near real time: spelling, punctuation, subject-verb agreement, tense, and for premium tiers, style and clarity rewrites.
How would you design this system? Cover the model architecture, the latency budget, the training data, and how you evaluate suggestion quality.

The Problem: suggestions must arrive faster than the next keystroke, and a wrong correction costs more trust than a missed error. Design the system behind the wavy underline.
Answer
The design is a model ladder behind a per-sentence router: cheap deterministic rules and dictionaries catch spelling and punctuation on-device, an edit-tagging Transformer (one parallel pass over the tokens emitting KEEP, DELETE, INSERT, REPLACE tags) handles most grammatical error correction in the cloud, a seq2seq model takes the sentences that need reordering, and an LLM serves style and clarity rewrites only on demand. The two pivotal decisions are routing each sentence to the cheapest model that can fix it (latency is a product feature), and tuning every error type to a precision-first operating point because false corrections erode trust faster than misses.
(1) Error Taxonomy Drives Architecture: mechanical errors (spelling, punctuation, agreement) and open-ended style rewrites are different problems and deserve different models.
(2) Model Ladder With Routing: rules → edit-tagger → seq2seq GEC → LLM, cheapest first; the router reads error type, confidence, and latency budget.
(3) Precision-First Operating Points: per-error-type thresholds tuned on F_0.5, since a wrong correction damages trust more than a miss.
(4) Latency Engineering: score only the current sentence, cache unchanged text, and keep the inline path under 100 ms end to end.
(5) Personalization & Dialect: locale (US/UK), register (casual vs formal), and per-user dismiss history modulate which suggestions fire.

Figure 1: The inline path: each finished sentence is routed to the cheapest sufficient model, and a merger dedupes overlapping suggestions before rendering underlines. Unchanged text is served from cache.
Clarify Before Designing:
(1) Scope: which languages and locales, and is style or clarity rewriting in scope or only mechanical correctness?
(2) Latency Budget: how many milliseconds for inline suggestions, and is there an async path for long-form checks?
(3) Deployment: what runs on-device versus in the cloud, given browser-extension and mobile constraints?
(4) Privacy: may user text leave the device, and are there regulated contexts (legal, medical) to special-case?
(5) Personalization: can we learn from per-user accept and dismiss behavior, and where may that profile live?
Login to view more content
August 8, 2026
MSD0009 Restaurant Recommendation System for Food Delivery

You are building the restaurant recommendation system for a food-delivery platform like Uber Eats or DoorDash. When a user opens the app, the home feed must rank nearby restaurants in a way that converts to orders, while respecting hard logistics: the restaurant must be open, inside the delivery radius, and able to deliver on time given the current courier supply.
How would you design this system? Cover the retrieval and ranking architecture, the features and objectives, and how you handle cold start for new restaurants.

The Problem: a million restaurants platform-wide, but only a few hundred can actually deliver to this user right now. Rank the right ones first, inside a lunch-rush latency budget.
Answer
The design is a multi-stage funnel with a hard logistics gate in front: a geo/open/ETA filter cuts the catalog to the few hundred restaurants that can actually serve the request, multi-source retrieval (two-tower embeddings, co-order graphs, trending-nearby) proposes candidates, a multi-task ranker scores them for order probability, basket value, and delivery time, and a re-ranker enforces cuisine diversity and new-restaurant exploration. The two decisions that make or break the system are treating deliverability as a constraint rather than a feature, and reserving exploration slots so new restaurants ever get seen.
(1) Funnel, Not One Model: filter → retrieve → rank → re-rank, so each stage stays cheap enough for the 50 ms feed budget.
(2) Logistics As Hard Filters: open hours, delivery radius, courier supply, and predicted ETA gate eligibility before any learned model runs.
(3) Context-Heavy Features: time of day, day of week, weather, and session activity matter as much as long-term taste, because lunch intent differs completely from late-night intent.
(4) Multi-Objective Ranking: combine P(order), expected basket value, and delivery-time risk into one score with explicit business weights.
(5) Cold Start By Construction: content-based embeddings give new restaurants a Day-1 representation, and reserved exploration slots guarantee them impressions.

Figure 1: The serving funnel: about one million restaurants shrink to roughly 300 deliverable candidates, then to a ranked top-20 feed. The dashed loop is the offline retraining path fed by logged impressions and orders.
Clarify Before Designing:
(1) Scale: how many cities, restaurants, and daily active users, and what is the peak feed QPS at lunch?
(2) Latency Budget: how many milliseconds for the home feed, and are personalized search results in scope too?
(3) Objective Priority: is the north star order conversion, gross merchandise value, or delivery-time reliability?
(4) Cold-Start Volume: how many new restaurants onboard per week, and in how many cities?
(5) Constraints: must we support sponsored slots, dietary filters, and per-market regulations?
Login to view more content
August 8, 2026
MSD0007 Demand Forecasting System for Retailer

Design a demand forecasting system for a large retail company like Costco, Walmart, or Target. The system should predict future product demand across stores and time to support inventory planning, replenishment, and promotions.

The Problem: thousands of SKUs across hundreds of stores. Predict each one’s future demand well enough to keep shelves full and waste low, then feed those numbers to replenishment and pricing.
Answer
The demand forecasting system ingests diverse data (sales, inventory, weather, promotions) and predicts product demand with ML models spanning time series, tree-based, and deep learning methods. It features a scalable layered architecture (data pipelines, real-time processing, integration for inventory management) and drives decisions with probabilistic forecasts, reducing stockouts, optimizing supply chains, and improving accuracy through iterative model training.
(1) Problem Definition & Metrics: Fix granularity (SKU-store-day), horizon (2-week operational, 3-month tactical), and objective (minimize out-of-stocks and waste); measure with WMAPE and forecast bias.
(2) Data Strategy: A unified feature store over internal signals (sales, hierarchy, pricing, promos, inventory, clicks) and external ones (calendar, weather, events, competitor, macro).
(3) Layered Architecture: Ingestion → processing & feature store → modeling layer → serving APIs → monitoring & feedback.
(4) Hierarchical Ensemble Modeling: Interpretable base (Prophet/ETS) at aggregate level, global LightGBM/XGBoost as the granular workhorse, TFT/DeepAR for high-volatility items, reconciled top-down and bottom-up.
(5) Probabilistic Forecasts: Output quantile forecasts (P10, P50, P90) so logistics can set safety stock from uncertainty, not a single number.

Figure 1: The five-layer system: ingestion and the feature store feed a model repository; forecasts are served through APIs to replenishment and pricing, while monitoring closes the loop back into retraining.
Clarify Before Designing:
(1) Scale: how many SKUs, stores, and series (SKU × store) must be forecast?
(2) Granularity & Horizon: store-day or region-week? 2-week replenishment or 3-month tactical planning?
(3) Decision Consumers: automated replenishment, or human planners reviewing suggestions?
(4) Data Reality: is a promotions calendar available in advance? New product launch cadence (cold-start rate)?
(5) Cost Asymmetry: the cost of a stockout (lost sale, empty shelf) versus overstock (waste, markdowns)?
Login to view more content
January 3, 2026
MSD0006 Video Recommendation System

How would you design a scalable and personalized video recommendation system for a platform like YouTube, Netflix, or TikTok that can recommend relevant videos in real time to billions of users?

The Problem: billions of videos, billions of users, one feed request at a time. Pick the top-20 most relevant videos for this user in milliseconds.
Answer
A modern recommendation system uses a multi-stage pipeline to narrow billions of videos down to a top-20 list for a user in milliseconds. It typically consists of Candidate Generation (filtering down to hundreds), Ranking (scoring those hundreds with deep learning), and Re-ranking (applying business logic for diversity, freshness, and safety). The full path is: Data Logging → Candidate Generation → Ranking → Re-ranking → Serving.
(1) Multi-Stage Pipeline: No single model can score a billion items per request. The funnel makes it affordable: retrieve ~500 candidates cheaply, then rank precisely, then polish.
(2) Data & Features: User signals (watch history, watch time, likes, skips, follows), video features (visual/audio/text embeddings, popularity, freshness), and context (time of day, device, network).
(3) Candidate Generation: Multi-source retrieval (collaborative filtering / two-tower, content-based, seed-based co-watch, trending), merged and deduplicated to ~100-500.
(4) Ranking: A deep model scores the candidates: DNNs on hundreds of features, Multi-Task Learning heads (click, watch time, like, share), and sequence models for session context.
(5) Re-Ranking: Business and quality constraints: remove seen/unsafe items, inject diversity and freshness, fight filter bubbles.

Figure 1: The recommendation funnel: each stage trades breadth for precision: cheap retrieval cuts billions to hundreds, the ranking model orders them precisely, and re-ranking applies the final business logic before serving.
Clarify Before Designing:
(1) Scale: how many videos, daily active users, and feed requests per second?
(2) Latency Budget: how many milliseconds for a feed request, end to end?
(3) Objectives: what are we optimizing (clicks, watch time, long-term satisfaction), and are there multiple conflicting goals?
(4) Cold Start Volume: how many new users and new videos arrive daily?
(5) Constraints: ads insertion, content safety, regional/legal rules that override relevance?
Login to view more content
January 2, 2026
MSD0004 Long Document Attention Scalability

The standard Transformer’s self-attention mechanism has a computational and memory complexity of $O(N^2)$ , where $N$ is the sequence length. For long document classification (e.g., thousands of tokens), this quadratic scaling becomes prohibitive.
Describe one or more attention modifications you would design or choose to enable efficient and effective long document classification.

The Problem: full self-attention costs $O(N^2)$ in compute and memory. Redesign attention so a several-thousand-token document stays affordable without giving up classification accuracy.
Answer
To handle long documents, the quadratic complexity of full self-attention ( $O(N^2)$ ) must be reduced. The two primary directions are Sparse Attention (Longformer, BigBird), which constrains each token to a relevant subset (a local window plus a few global tokens), and Hierarchical Attention (HATN), which applies attention within segments first and then across segment representations. Both preserve classification quality while cutting cost from $O(N^2)$ toward linear; linearized and recurrent variants (Performer, Linformer, Transformer-XL) attack the same wall from other angles.
(1) Sparse Attention (Mechanism): Replace the full attention matrix with a sparse design: local window attention for nearby context plus global tokens (e.g., [CLS]) that exchange information with all tokens (Longformer, BigBird).
(2) Hierarchical Attention (Structure): Split the document into segments; apply token-level attention within each segment, then a document-level attention across segment representations (HATN, LNLF-BERT).
(3) Linearized / Recurrent Variants: Kernel approximations (Performer, Linformer) reach true linear complexity; Transformer-XL / Compressive Transformer carry a recurrent memory across segments.

Figure 1: Full attention (left) fills the whole $N \times N$ matrix; sliding-window attention (right) keeps only a diagonal band: each token attends to its neighbors, and cost drops from $O(N^2)$ to $O(N \cdot w)$ .
Clarify Before Designing:
(1) Document Length: typical and maximum token count: 2k, 8k, 32k?
(2) Starting Point: adapt a pretrained BERT-style encoder, or train from scratch?
(3) Binding Constraint: memory at training, or latency at inference?
(4) Interaction Pattern: does classification need global token interactions (a [CLS] readout), and how long-range are the dependencies?
(5) Budget For Change: may we modify the architecture, or only fine-tune an off-the-shelf long model?
Login to view more content
November 4, 2025
MSD0003 Spam Email Detection

Design an end-to-end Machine Learning system to effectively detect and filter spam emails in a high-volume email service.
Describe how you would design, train, and deploy this system.

The Problem: a high-volume mail stream must be split into inbox and spam in real time, where one false positive can lose a user’s important email forever.
Answer
The system is a real-time classification pipeline at the mail gateway. It begins with data collection and preprocessing (features from text and metadata) and trains a supervised model on user-labeled history. Deployment is a low-latency prediction service that scores every incoming email and applies a tuned threshold. Because spammers constantly adapt, the system lives or dies by its monitoring and continuous-training loop that fights concept drift.
(1) Objectives & Metrics: Precision first: a false positive (legitimate mail in spam) is far costlier than a false negative; keep recall high as the secondary goal and track F1 / PR-AUC.
(2) Data & Features: User-labeled history plus external corpora; features span email text (subject, body), metadata (sender, domain reputation, recipients), and behavior (links, attachments, frequency).
(3) Two-Stage Cascade: A fast traditional model filters the obvious ~80%, then a Transformer deep-analyzes the ambiguous ~20%: accuracy where it matters, speed everywhere else.
(4) Deployment: A low-latency prediction API at the gateway: receive → extract features → score → threshold → route to spam folder or inbox.
(5) Maintenance: Monitor precision/recall daily, harvest user feedback (‘Mark as Spam’ / ‘Not Spam’) as fresh labels, and retrain continuously against concept drift.

Figure 1: The real-time inference path: every incoming email is scored by the model (e.g., 0.95), and a tuned threshold (e.g., 0.8) routes it (spam folder or inbox) inside the latency budget of the mail gateway.
Clarify Before Designing:
(1) Scale: email volume per day and peak QPS at the gateway?
(2) Latency Budget: how many milliseconds may scoring add before the email is delivered?
(3) Cost Asymmetry: confirmed: is a false positive (lost legitimate mail) really the catastrophic error here?
(4) Labels: how much user-labeled spam/not-spam data exists, and how fresh is it?
(5) Adversaries: how quickly do spam tactics mutate in this ecosystem: days, weeks?
Login to view more content
November 2, 2025