Interview for Machine Learning

Author: admin

MSD0054 Warehouse Robot RL Pathfinding
Design a deep reinforcement learning system for warehouse robot path planning. An e-commerce fulfillment center (Amazon Robotics, Ocado) operates hundreds of mobile robots that navigate warehouse aisles to retrieve pods, transport goods to packing stations, and return to storage locations. Each robot must plan a collision-free path to its goal while avoiding static obstacles (shelves, pillars), dynamic obstacles (other robots, humans), and deadlocks where two robots block each other in a narrow aisle. The warehouse is a structured but dynamic grid: shelf layouts change as inventory moves, traffic density varies by time of day and order volume, and a single blocked aisle can cascade into gridlock across the entire floor. A deep RL system can learn coordinated path planning policies that outperform classical A* or Dijkstra-based planners in dense, multi-agent settings, but it must be trained safely (without crashing real robots), generalize to warehouse layouts it has never seen, and handle the non-stationarity of a live fulfillment center.
How would you design this system? Cover the RL formulation (state, action, reward design for goal-reaching vs collision-avoidance vs throughput), the candidate RL architectures (centralized training with decentralized execution vs multi-agent PPO vs graph neural network-based coordination), how you train safely in simulation and transfer to real warehouses (sim-to-real), how you handle non-stationary traffic and layout changes, and how you evaluate path quality and warehouse throughput against classical planners.

The Problem: hundreds of robots share narrow aisles, the shelf map changes with inventory, and one pair of robots blocking each other can stall the whole floor. Design the planner that keeps every robot moving toward its goal without a single collision.
Answer
The design is a hybrid planner: a classical shortest-path heuristic handles the global map, a learned per-robot policy handles the local, multi-agent decision of which move to take right now, and a hard safety shield sits between the policy and the motors. The policy is trained with centralized training, decentralized execution (multi-agent PPO with one shared actor and a team-level critic) inside a randomized simulator that never touches a real robot. The three pivotal decisions are to give the policy a precomputed path heuristic as an input channel so RL never has to rediscover the floor plan, to make every learned action pass a reservation-table and deadlock check before actuation, and to roll out through shadow mode and zone canaries where the classical planner keeps executing until the learned policy has earned trust.
(1) Hybrid Formulation: RL owns the local decision (one of five moves given a 9×9 egocentric window and neighbors’ intended next cells), while a per-map shortest-path distance field enters as an observation channel; layout changes become a heuristic recompute, not a retrain.
(2) CTDE With MAPPO: one shared actor runs on every robot, and a centralized critic sees the whole fleet during training only; shared weights make the robot count a runtime parameter rather than an architecture choice.
(3) Reward Shaping: potential-based progress on shortest-path distance, collision and idle penalties, and a team throughput bonus; the mix is tuned in simulation against measured throughput, never hand-tweaked on the live floor.
(4) Safety Shield: every proposed action is checked against a reservation table and a wait-for-graph deadlock detector; the policy proposes, the shield disposes, so collision-free is a property of the shield, not a hope about the network.
(5) Randomized Simulation and Staged Rollout: procedurally generated layouts, replayed order streams, and injected comms delay train a policy that generalizes; shadow mode, a single-zone canary, and rollback gates carry it to the floor.

Figure 1: Train in simulation, execute per robot behind a shield: the learned policy only ever proposes a move, and real fleet logs flow back to keep the simulator honest.
Clarify Before Designing:
(1) Fleet and Floor: how many robots on how many square meters, and is the floor a fixed grid (Ocado-style hive) or free aisles with pods (Kiva-style pods)?
(2) Control Topology: does a central fleet server talk to every robot at a fixed tick, or must each robot decide on-board with lossy wireless; this decides whether decentralized execution is a choice or a necessity.
(3) Humans on the Floor: is the space caged robot-only, or mixed with pickers (as with Amazon’s Proteus); mixed traffic changes the safety envelope and the observation design.
(4) Layout Churn: how often does the map version change (daily inventory moves vs quarterly re-slotting), and is a new map available before robots see it?
(5) Latency Budget: what is the decision tick (100 ms is typical), and how much of it can planning consume after sensing and communication?
(6) Baseline and Success: what does the current planner achieve in trips per hour and stalls per thousand trips, and is task assignment (which robot takes which order) in scope or fixed upstream?
Login to view more content
September 8, 2026
MSD0053 Video Chaptering Keyframe
Design a deep learning system for autonomous video chaptering and keyframe extraction. A video platform (YouTube, Vimeo, Coursera) must automatically segment uploaded videos into chapters and extract representative keyframes so that viewers can navigate to specific sections without scrubbing through the entire video. For a 45-minute lecture, the system must detect topic boundaries (when the speaker moves from “gradient descent” to “backpropagation”), assign a chapter title to each segment, and pick one keyframe per chapter that best represents its content.
The system must handle diverse video types (lectures, vlogs, tutorials, interviews, gameplay) with no manual metadata, detect both visual and audio scene changes (a slide change, a speaker cut, a topic shift in the transcript), and produce chapter titles that are concise and informative. The system must process hundreds of thousands of uploaded videos per day, and the chaptering must be good enough that viewers prefer the auto-generated chapters over no chapters at all.
How would you design this system? Cover the multimodal feature extraction (visual embeddings from frame sampling, audio features, transcript text), the boundary detection model (shot boundary detection vs topic segmentation vs learned segmentation), the keyframe selection strategy (clustering vs diversity sampling vs attention-based), the chapter title generation approach (extractive from transcript vs abstractive summarization vs LLM generation), and how you evaluate chaptering quality when there is no ground truth for most videos.

The Problem: a 45-minute video arrives as raw frames and audio with no table of contents. Find where each topic starts, name it, and pick the one frame that stands for it, for hundreds of thousands of uploads a day.
Answer
The design is an upload-time batch pipeline that turns every video into a per-second sequence of multimodal tokens (frame embeddings with OCR text, speaker-change and music cues from the audio, sentence embeddings from the ASR transcript) and runs a learned boundary transformer over that sequence to predict where chapters start. Shot-boundary detection and transcript topic segmentation are input features to this model rather than the model itself, so a slide flip alone does not cut a chapter while a topic shift inside one static shot still does. The model is trained on creator-supplied chapter timestamps, the millions of videos where creators already typed chapters into the description, filtered to remove noisy chaptering. Each chapter then receives a keyframe chosen as the quality-filtered medoid of its frame embeddings and a title written by an instruction-tuned LLM that reads the chapter’s transcript and slide text under a grounding check, with an extractive fallback. Quality is proven the way the product will be judged: tolerance-window F1 against creator chapters, pairwise human ratings, and online navigation behavior.
(1) One Time Axis: resample every modality to one token per second so boundaries, keyframes, and titles share coordinates; a 45-minute lecture becomes 2,700 tokens.
(2) Weak Labels from Creators: train on creator timestamps that satisfy the platform’s chapter rules (at least three chapters, each 10 s or longer, first at 0:00), reweighted by video type and held out by creator.
(3) Learned Boundary Model over Signals: shot cuts, speaker changes, and transcript cohesion dips are features; a transformer outputs P(boundary at t) and decoding enforces a minimum chapter length.
(4) Quality-Filtered Medoid Keyframe: discard blurry, dark, and transition frames near boundaries, then pick the medoid frame, the one that minimizes total distance to every other frame in the chapter.
(5) Grounded LLM Titles: generate 2-6 word titles from the transcript plus OCR text in one call per video; reject titles whose content words never appear in the source and fall back to the top extractive phrase.
(6) Evaluation Without Truth: tolerance-window boundary F1 on creator-chaptered holdouts, pairwise raters, and chapter click and seek metrics online; withhold chapters when confidence is low.

Figure 1: The chaptering pipeline: three feature streams on one time axis, one learned boundary model, then keyframe and title stages per chapter, with creator edits and navigation logs feeding retraining.
Clarify Before Designing:
(1) Content Mix and Length: what share of uploads is speech-dense (lectures, interviews) versus gameplay, music, or silent screen recordings, and what does the duration tail look like (multi-hour stream replays)?
(2) Freshness Budget: must chapters exist at publish time, or is a delay of minutes to an hour acceptable, and may we reprocess the back catalog when the model improves?
(3) Existing Signals: does the platform already run ASR for captions, and does it store creator-typed chapter timestamps that can serve as labels?
(4) Product Constraints: minimum and maximum chapter counts, minimum chapter length, title length cap, and whether creators can edit or disable auto chapters?
(5) Cost Envelope: how many GPU-hours per thousand video-hours are allowed, which decides frame sampling rate, ASR model size, and whether an LLM runs per video?
(6) Language Coverage: which languages need titles, and how reliable is the transcript for each?
Login to view more content
September 8, 2026
MSD0052 Airbnb Photo Tagging
Design a system that automatically tags photos uploaded by hosts (e.g., “kitchen”, “bedroom”, “pool”, “ocean view”) on Airbnb using a deep image classification model. When a host creates or edits a listing, they upload dozens of photos, and Airbnb must automatically label each photo with the room type or amenity it depicts so that guests can filter by feature (“pool”, “mountain view”) and the listing page can organize photos into structured sections (Living Room, Bedroom, Kitchen, Exterior). The system must handle a wide taxonomy of tags (50+ room types and amenities), work across varying photo quality, angles, and lighting from phone cameras worldwide, distinguish similar categories (“living room” vs “family room”, “balcony” vs “patio”), and handle the long tail of rare amenities (“sauna”, “wine cellar”) that have few training examples. Mislabeling a bathroom as a kitchen degrades search relevance and user trust, so the system must flag low-confidence tags for host confirmation rather than auto-assigning them.
How would you design this system? Cover the classification model architecture (single-label vs multi-label vs hierarchical classification), how you handle the long tail of rare tags with few training examples, how you generate training labels from host-provided metadata and guest clicks, how you calibrate confidence for the host-confirmation workflow, the inference pipeline for processing photo uploads in real time, and how you evaluate tagging accuracy across the full tag taxonomy.

The Problem: Thirty phone photos arrive per listing and each must land on one of fifty-plus tags, where the popular tags have millions of examples, the rare ones have a few hundred, and a confident wrong tag costs more trust than no tag at all.
Answer
The design is a shared vision backbone with two heads: a hierarchical softmax head for the mutually exclusive room or scene type, and a multi-label sigmoid head for amenities that co-occur in one photo. Training labels are mined at scale from host captions, existing photo-section placement, and guest filter clicks, then sharpened by the host confirmations the product itself generates. Rare tags are handled by decoupling representation from classifier: the backbone learns from the head-heavy data, the classifier heads are re-balanced, and tags with too few examples are served through a nearest-prototype path until they graduate. Every prediction passes a per-tag calibrated confidence gate with three outcomes: auto-assign, a one-tap host confirmation card, or leave untagged. Inference runs as a batched GPU service triggered by the upload event, so a 30-photo listing is tagged before the host finishes the flow, and every confirmation flows back into the next training round.
(1) Label Structure: a scene tree (indoor, outdoor, view) with roughly 40 leaves under a hierarchical softmax, plus roughly 30 amenity tags under independent sigmoids; sibling pairs humans confuse (“living room” vs “family room”) share one merged leaf until a reliable cue separates them.
(2) Weak Labels at Scale: host captions, section placement, listing-level amenity declarations, and guest clicks after filtering by an amenity give millions of noisy labels; host confirmations and removals form the small gold layer that drives thresholds and evaluation.
(3) Long Tail by Decoupling: train the backbone on all data, then re-fit the classifier heads with class-balanced sampling and logit adjustment; tags below about 5k labels use a prototype classifier on frozen embeddings.
(4) Calibrated Confidence Gate: temperature scaling per head and per-tag thresholds tuned on the confirmed holdout to a 98% precision floor; high confidence auto-assigns, the gray zone asks the host, low confidence stays untagged.
(5) Async GPU Serving at Upload: an upload event triggers fetch, dedupe, batch inference, and a write to a versioned tag store that feeds search filters and section layout; the catalog backfill is a batch job on the same model.
(6) Confirmation Feedback Loop: confirmations, removals, and re-sectioning are logged as labels with model version, and monthly retraining passes per-region and per-tag gates before release.

Figure 1: One backbone pass per photo, two heads, one gate: confident tags ship, doubtful ones become a host card, and every host answer becomes a training label.
Clarify Before Designing:
(1) Taxonomy Ownership: is the 50-tag list fixed or does the product team add tags quarterly, and does a hierarchy (room vs amenity vs view) already exist?
(2) Latency Expectation: must tags appear inside the upload flow within seconds, or only before the listing goes live minutes later?
(3) Label Sources: what fraction of photos carry a host caption or a section placement, and are guest filter-and-click logs joinable to individual photos?
(4) Error Asymmetry: which hurts more, a missing “pool” or a false “pool”, and is there a precision floor the search team already enforces?
(5) Scale: photos uploaded or edited per day, catalog size for a backfill, and the share of phone versus professional photos?
(6) Confirmation Budget: how many confirmation prompts per upload before hosts disengage, and may a listing publish while tags are pending?
Login to view more content
September 8, 2026
MSD0051 Street View Face Blurring
Design the face and license-plate blurring system for Google Street View using a deep object detection model. Street View captures panoramic street-level imagery from cameras mounted on cars, trekkers, and drones, covering millions of miles of roads across 100+ countries. Every captured panorama must be processed to blur all human faces and vehicle license plates before publication, to comply with privacy laws (GDPR in Europe, CCPA in California, and local regulations worldwide). The system must detect faces and plates at varying scales, angles, lighting, and occlusion (a face in a car window, a plate partially covered by snow), process millions of panoramas per day with minimal compute cost, and keep the false-negative rate near zero because a single unblurred face is a privacy violation. At the same time, false positives that blur non-face objects (signs, manhole covers, building facades) degrade image quality and user experience.
How would you design this system? Cover the detection model architecture (single-stage detector vs two-stage detector vs segmentation-based), how you handle the extreme scale variation and small objects, how you generate and augment training data for rare cases (motorcycle plates, faces in helmets), how you balance the false-negative vs false-positive trade-off given the privacy-first requirement, the inference pipeline for processing millions of panoramas per day, and how you monitor and audit blur quality at scale.

The Problem: every panorama hides a handful of faces and plates at 20 to 200 pixels among signs and facades that look like them; miss one and it is a privacy violation, blur too many and the street turns to mush.
Answer
The design is a recall-first single-stage detector run on overlapping tiles of the full-resolution panorama, feeding a fail-closed blur gate that no image can bypass on its way to publication. An FPN-style anchor-free detector with separate face and plate heads scores every 1024 px tile at native resolution, detections are merged with panorama-level NMS, and the threshold is set far below the mAP-optimal point so recall, not precision, is what the model is tuned for. Boxes are dilated before blurring, low-confidence boxes go to a cheap verifier that promotes rather than vetoes, and a stratified human audit plus the user report path close the loop on the misses no model catches. The pivotal calls are tiling instead of downscaling, an explicit blur budget instead of a precision target, and treating the blur job as a publication gate rather than a post-processing step.
(1) Tiled Native-Resolution Input: the 8k by 4k panorama is cut into overlapping 1024 px tiles so a 20 px face is still 20 px when the detector sees it; a 2x upsampled re-crop handles the smallest bucket.
(2) Single-Stage FPN Detector: an FCOS-class anchor-free model with focal loss, separate face and plate heads, and pyramid levels matched to the 16 to 256 px object range.
(3) Recall-First Operating Point: the threshold is chosen from the recall-versus-false-positives-per-panorama curve, with a blur budget (blurred pixel fraction) as the only cap on precision loss.
(4) Blur If In Doubt: boxes are dilated 15 to 20%, overlapping boxes are unioned, and an uncertain band is re-scored by a crop verifier that can add blur but never remove a confident detection.
(5) Targeted Data Program: country-stratified plate labels, hard-negative mining on facades and signs, and copy-paste augmentation for motorcycle plates, helmets, masks, and faces behind glass.
(6) Fail-Closed Batch Gate: a panorama publishes only with a signed blur receipt; stratified human audit and user reports feed misses and false blurs back into retraining.

Figure 1: The publication path: tile, detect, merge, blur generously, and never publish without a blur receipt; the audit loop underneath is what keeps recall honest over time.
Clarify Before Designing:
(1) Scale and Latency: how many panoramas per day and how many pixels each; is publication latency hours or days, which decides whether this is a batch system or a streaming one?
(2) Legal Bar: which jurisdictions and what counts as a face or plate (statues, posters, reflections, partial plates), and is “near zero” a measured audit rate or a contractual guarantee?
(3) Blur Budget: what fraction of blurred non-target pixels per panorama is acceptable before image quality complaints outweigh privacy gains?
(4) Capture Fleet: how many camera generations and platforms (car, trekker, drone) with different resolutions and viewpoints must one model serve?
(5) Data Assets: do labeled faces and plates from earlier campaigns exist, and can already-published blurred imagery be reused as training signal?
(6) Back-Catalog: must a better model reprocess years of published imagery, which changes the compute plan and the rollout policy?
Login to view more content
September 8, 2026
MSD0050 Document QA Extraction
Design a question-answering system that extracts an answer from a large document collection given a user query using a deep reading comprehension model. A legal or enterprise search product must let users ask natural-language questions (“What is the termination clause in the NDA?”, “What were the Q3 revenue figures for the EMEA region?”) and return exact answer spans extracted from a corpus of millions of documents: contracts, filings, internal wikis, emails.
The system must first retrieve the relevant documents or passages from the corpus (the retrieval stage), then run a reading comprehension model over the retrieved passages to extract the answer span (the reader stage), and finally rank or aggregate answers when multiple passages produce candidates. The retrieval stage must handle lexical mismatch (the query says “termination” but the document says “cancellation”), the reader must handle multi-hop reasoning where the answer requires combining information from two passages, and the system must say “I don’t know” when no passage contains a confident answer rather than hallucinating.
How would you design this system? Cover the retriever architecture (sparse BM25 vs dense bi-encoder vs hybrid retrieval), the reader architecture (extractive span prediction vs generative reading comprehension vs T5/BART-style seq2seq), how you handle multi-hop questions, how you calibrate confidence for abstention, and how you evaluate end-to-end QA accuracy (exact match, F1, answerability detection).

The Problem: one natural-language question, ten million documents that rarely use the asker’s words, and a product that must quote the exact sentence or admit it cannot find one.
Answer
The design is a retrieve-rerank-read pipeline with an explicit abstention gate. A hybrid retriever (BM25 plus a dense bi-encoder, merged by reciprocal rank fusion) pulls about 100 passages from a permission-filtered index, a cross-encoder reranker keeps the top 10, and an extractive reader with a no-answer head scores candidate spans in each passage. Spans are aggregated across passages, and a calibrated confidence threshold decides between returning a cited span and answering “I don’t know”. Multi-hop questions run the same loop twice: the first hop’s reader output becomes a bridge entity that rewrites the query for a second retrieval. The pivotal decisions are hybrid rather than pure dense retrieval (synonyms and exact identifiers both matter in legal and financial text), extractive rather than generative reading (verbatim spans with citations are the product, and a span cannot hallucinate), and a threshold set on the risk-coverage curve rather than on a raw softmax.
(1) Passage Index with Metadata: split documents into 200-300 token passages with overlap, and store document type, date, region, and access-control lists alongside both the inverted index and the ANN index so filters apply at retrieval time.
(2) Hybrid Retrieval: BM25 catches exact identifiers (clause numbers, “Q3”, “EMEA”) while the dense bi-encoder catches paraphrase (“termination” versus “cancellation”); reciprocal rank fusion merges both lists into a top-100.
(3) Cross-Encoder Reranker: joint query-passage encoding narrows 100 → 10 passages and is the largest single quality gain per millisecond in the pipeline.
(4) Extractive Reader with Null Head: start and end logits plus a no-answer score per passage; identical spans from different passages pool their probability.
(5) Iterative Multi-Hop: when hop one yields a bridge entity but no final answer, the query is rewritten and retrieval runs again, capped at two hops to bound latency.
(6) Calibrated Abstention: the null-versus-span margin threshold is chosen on the risk-coverage curve at the product’s tolerated error rate, and abstentions fall back to classic search or a human.

Figure 1: The retrieve-rerank-read path with its two exits, a cited span or an honest abstention, and the dashed multi-hop loop that rewrites the query with a bridge entity.
Clarify Before Designing:
(1) Corpus and Freshness: how many documents and pages, in which languages and formats (scanned PDFs need OCR), and how quickly must a new filing become searchable?
(2) Load and Latency: queries per day, acceptable p95 latency for an interactive answer, and whether a slower two-hop path is tolerable for a minority of questions?
(3) Answer Form: is the product a verbatim span with a citation, or a synthesized paragraph; may the system quote two passages for one answer?
(4) Multi-Hop Share: what fraction of real questions need two passages, and is a bridge through an entity (a counterparty, a region, a fiscal quarter) the dominant pattern?
(5) Labels: do in-domain question-answer pairs exist (support tickets, FAQ pages, past research memos), and what annotation budget do domain experts have?
(6) Error Asymmetry and Access: how much worse is a confidently wrong clause than “I don’t know”, and must answers respect per-user document permissions?
Login to view more content
September 8, 2026
MSD0049 Google Related Searches
When a user enters a query on Google, the search results page shows a list of “related searches” at the bottom: queries that other users have issued in the same session, or queries that are semantically close to the original. These suggestions help users refine their intent, discover alternative phrasings, and navigate to adjacent topics, and they drive a significant fraction of query reformulations. Generating them at scale requires a deep embedding model that maps queries into a semantic space where nearby queries are related, plus a retrieval mechanism that can surface the best candidates from billions of historical queries in milliseconds. The model must handle the long tail of rare queries that have little or no co-occurrence history, distinguish genuinely related queries from merely popular ones, and avoid suggesting queries that lead to harmful or policy-violating content.
How would you design this model? Cover the candidate embedding architectures (dual-encoder trained on query co-occurrence vs contrastive learning on query-query pairs vs LLM-based query expansion), how you build the retrieval index over billions of queries, how you handle the long tail of rare and unseen queries, how you filter harmful or policy-violating suggestions, and how you evaluate related-search quality (click-through rate, reformulation success rate, relevance judgments).

The Problem: billions of past queries could fill the three empty slots under the results, most of them were typed only once, some of them should never be shown, and the choice must be made in a few milliseconds.
Answer
The design is a contrastive query encoder: a BERT-base transformer (110M, 256-d output projection) trained with InfoNCE on query-query pairs harvested from session reformulations and same-click queries, using in-batch negatives plus mined hard negatives and a logQ popularity correction so that cosine similarity means “related” rather than “popular”. Every historical query is embedded once offline into an approximate nearest-neighbor index, and at serve time the live query is encoded and its neighbors are pulled out and filtered. LLM query expansion is deliberately not the online model. It runs offline to synthesize and relabel training pairs and to generate grounded candidates for tail queries that have no usable neighbors. The two pivotal decisions are to train on semantic pairs with explicit popularity debiasing instead of factorizing raw co-occurrence counts, and to keep the LLM offline and grounded so that every displayed suggestion is a real, auditable query.
(1) Dual-Encoder on Co-occurrence: two towers whose training target is the session co-occurrence matrix; a sampled softmax predicts the next or co-session query, which effectively factorizes the count matrix into embeddings.
(2) Contrastive Query-Query Encoder: one shared transformer trained with InfoNCE on curated positive pairs (reformulations, same-click queries, paraphrases), in-batch and mined hard negatives, a temperature, and a logQ correction; retrieval is cosine k-NN over the index.
(3) LLM-Based Query Expansion: a generative model writes related queries directly from the query text, or synthesizes and labels pairs for distillation; generated strings are snapped to real historical queries before use.

Figure 1: The contrastive query encoder is trained once on pairs, embeds the whole historical corpus offline, and serves each live query with one cheap forward pass plus a nearest-neighbor lookup.
Clarify Before Designing:
(1) Candidate Pool: must every suggestion be a real historical query (grounded, auditable), or may the system show generated strings nobody has typed?
(2) Latency and Compute Budget: how many milliseconds and FLOPs per query for encode plus retrieve, and may head queries be precomputed and cached?
(3) Training Signal: which logs exist at what volume: session reformulations, same-click query pairs, human relevance judgments; how many languages?
(4) Definition of Related: refinements, broadenings, lateral topics, or all three, and how much diversity is wanted versus precision?
(5) Tail Share: what fraction of distinct queries and of traffic is seen fewer than a handful of times, and must those queries receive suggestions or is an empty slot acceptable?
(6) Safety Bar: which policy categories apply (health misinformation, harassment, adult content, queries about private individuals), and is any false negative tolerable?
Login to view more content
September 8, 2026
MSD0048 Duolingo Story Difficulty Recommender
Duolingo wants to recommend stories of increasing difficulty as a student learns a language. The app has a library of short reading passages across multiple languages, each with a CEFR level (A1 to C2), and it must serve each learner a story that stretches them just past what they can already read comfortably — challenging enough to push growth but not so hard that the learner gives up. The core challenge is measuring story difficulty and learner ability jointly: a story’s difficulty depends on vocabulary frequency, grammar complexity, sentence length, and concreteness, while a learner’s ability depends on which words and grammar they have mastered so far. Deep knowledge tracing (DKT) can model a learner’s evolving knowledge state from their exercise interactions (correct/wrong, hints used, time spent), and this state can be matched against story difficulty estimates to pick the next story.
How would you design this model? Cover the candidate knowledge tracing architectures (deep knowledge tracing with recurrent networks vs transformer-based knowledge tracing vs item response theory enhanced with neural features), how you estimate story difficulty from text features and historical learner performance, how you combine the learner state with story difficulty to produce a recommendation score, how you handle cold start for new stories and new learners, and how you evaluate recommendation quality beyond simple accuracy.

The Problem: the shelf is sorted by CEFR level, but the learner is not a CEFR level. Which story is one step past what this particular learner can already read, and how would a model know?
Answer
The design is an attention-based knowledge tracer with an IRT-shaped head. A causal transformer reads the learner’s interaction history (which exercise, correct or not, hints used, response time) and emits two outputs: a scalar ability $\theta_t$ on a logit scale and a per-skill mastery vector over the words and grammar points the course teaches. Each story is placed on the same logit scale with a difficulty $\beta_s$ that starts from text features (vocabulary frequency, sentence length, clause depth, concreteness) and is then calibrated on historical learner outcomes. The recommendation score is the predicted comprehension probability $\sigma(\theta_t - \beta_s)$ pulled toward a target band of 0.70 to 0.85, plus a bonus for stories whose skills sit in the learner’s learning zone. The two pivotal decisions are to put ability and difficulty on one shared scale, so a brand-new story can be placed from its text alone, and to window the attention so the per-request cost stays flat as histories grow.
(1) DKT With Recurrent Networks: an LSTM consumes embeddings of (skill, correctness, hint, time) per interaction; its hidden state is the knowledge state and a sigmoid layer reads out per-skill mastery.
(2) Transformer Knowledge Tracing: SAKT, SAINT, or AKT-style causal attention in which the next item queries the interaction history; a distance-aware decay supplies recency and Rasch-style item embeddings supply a difficulty scale.
(3) IRT Enhanced With Neural Features: the Rasch logistic $\sigma(\theta - \beta)$ stays as the output model, while neural front-ends estimate $\theta_t$ from history and $\beta_s$ from story text (Deep-IRT, knowledge tracing machines).

Figure 1: Two estimators on one logit scale: the learner side traces ability and mastery from interactions, the story side places difficulty from text and outcomes, and the match picks the story whose predicted comprehension lands in the target zone.
Clarify Before Designing:
(1) Skill Granularity: is mastery tracked over individual words (thousands per course) or over grammar concepts and word families (hundreds)? This sets the output width and how sparse each skill’s evidence is.
(2) History Length: how many interactions does an active learner accumulate, hundreds or tens of thousands, and must the model attend to all of it or only a recent window?
(3) Inference Budget: is the learner state updated online after every exercise on a CPU budget of a few milliseconds, or recomputed in a nightly batch?
(4) Outcome Signal: do stories carry comprehension checks, or only completion and abandonment, and are hints and response times logged reliably enough to be features?
(5) Per-Language Data: how many learners does the smallest course have, since that number decides whether a transformer is even trainable there?
(6) Difficulty Ground Truth: are the editorial CEFR tags labels the model must respect, or priors it may revise from learner outcomes?
Login to view more content
September 8, 2026
MSD0047 Robinhood Trading Fraud Detection
Design Robinhood’s real-time trade monitoring system that analyzes order book activity to identify spoofing, wash trading, and insider trading patterns. A retail brokerage must monitor millions of orders per day in real time, detecting manipulative patterns the moment they form so that offending accounts can be flagged, restricted, or reported to regulators (SEC, FINRA).
Spoofing means placing large orders with no intent to execute, solely to move the price, then canceling them. Wash trading means an account trading with itself to create fake volume. Insider trading means unusual trading activity correlated with material non-public information. Each pattern has a distinct signature in the order book, but the signals are noisy, the patterns evolve as manipulators adapt, and false positives that freeze legitimate accounts damage trust and invite regulatory scrutiny.
How would you design this system? Cover the real-time order book feature engineering, the detection model architecture for each manipulation pattern, how you handle the adversarial evolution of manipulation tactics, the alerting and human-review workflow, and how you measure detection precision and recall when confirmed manipulation labels are scarce.

The Problem: three very different manipulation signatures hide inside tens of millions of daily order events, and each one has to be recognized while it is still forming.
Answer
The design is a streaming surveillance pipeline with one detector per manipulation pattern on top of a shared order-lifecycle feature layer. Every order event (new, modify, cancel, fill) lands in an append-only log, a stateful stream processor maintains per-account and per-symbol rolling features, and three specialized models consume them: a sequence model for spoofing and layering, a graph model over the account-to-account fill network for wash trading, and a retrospective event-study anomaly detector for insider trading. A fusion layer ranks candidates and thresholds them against actual analyst capacity, not against a fixed probability. The two pivotal decisions are to treat scarce labels as a modeling problem in their own right (weak supervision, positive-unlabeled learning, injected synthetic manipulation for recall estimation) and to keep probabilistic models out of the pre-trade order path, where only deterministic checks such as self-trade prevention belong.
(1) Shared Order-Lifecycle Features: compute cancel-to-fill ratio, order lifetime, depth added within N ticks, book imbalance, and contra-side fill proximity once per event and reuse them across all detectors.
(2) One Detector Per Pattern: the three signatures live in different data structures (temporal sequences, an account graph, price residuals around news), so a single “manipulation” classifier would blur them and lose explainability.
(3) Label-Scarce Learning: a few dozen confirmed cases cannot train a supervised model alone, so combine weak supervision from analyst dispositions and regulatory case patterns with an anomaly floor that catches novel tactics.
(4) Capacity-Calibrated Tiering: thresholds are set so that the daily alert volume matches reviewer throughput, with only the highest-precision deterministic patterns eligible for automated restriction.
(5) Adversarial Maintenance: champion-challenger models, red-team simulation of evasion (order splitting, cross-account layering), and drift monitors on the feature distributions manipulators can move.
(6) Auditability By Construction: every alert is reproducible from the immutable event log with a feature snapshot and reason codes, because a FINRA inquiry asks why a specific account was flagged on a specific day.

Figure 1: One feature layer, three pattern-specific detectors, one capacity-calibrated queue, and a feedback loop that turns analyst dispositions back into labels.
Clarify Before Designing:
(1) Regulatory Scope: which patterns are we obligated to surveil, and does the output feed formal reporting (SAR, SRO referral) or only internal risk decisions?
(2) Intervention Point: must we block a manipulative order before it reaches the venue, or is flagging within seconds after the fact acceptable?
(3) Data Reach: do we see only our own order flow, or consolidated cross-venue data, since a spoofer can place the phantom side elsewhere?
(4) Label Supply: how many confirmed cases exist historically, and do regulators tell us whether a referral was substantiated?
(5) Review Capacity And Cost Asymmetry: how many alerts per day can analysts actually clear, and what is the cost of freezing a legitimate account versus missing a spoof?
(6) Account Universe: retail only, or also market makers and institutions whose high cancel ratios are entirely legitimate?
Login to view more content
September 3, 2026
MSD0046 Multi-Camera Tracking Surveillance
Design a multi-camera vehicle and person tracking engine that fuses feeds from dozens of overlapping cameras across a campus, parking lot, or city intersection into a unified, persistent track for every vehicle and pedestrian. Each camera sees only a partial view: objects enter and leave frames, get occluded by each other and by structures, change appearance across lighting and weather, and must be handed off from one camera to the next as they cross camera boundaries.
The engine must run in real time across all cameras, maintain consistent global IDs across views (the same car is ID 42 in camera 3 and camera 7), recover from occlusions and re-identification failures, and scale to hundreds of cameras without linearly growing compute.
How would you design this model? Cover the candidate multi-object tracking architectures (per-camera tracking plus cross-camera re-identification, joint multi-camera tracking, tracking-by-detection with a global association layer), the re-identification feature design and how it survives viewpoint and lighting change, the cross-camera handoff and global ID assignment strategy, how you handle occlusion and track fragmentation, and how you scale inference across many cameras.

The Problem: every camera invents its own local track ID, and the same vehicle looks different in every view. The model’s job is to decide which local tracks are one object and keep that decision stable for hours.
Answer
The design is tracking-by-detection with a two-tier association: a cheap per-camera tracker turns detections into short, high-purity tracklets, and one global association layer stitches tracklets from all cameras into persistent global IDs. Two decisions are pivotal. First, associate at the tracklet level, not the detection level, which shrinks the matching problem by one to two orders of magnitude and gives ReID a multi-frame embedding instead of one blurry crop. Second, gate every candidate pair through a camera-topology graph with travel-time feasibility, so cost grows linearly in cameras rather than quadratically. Geometric bird’s-eye-view fusion is used only where calibrated overlap actually exists, because most real deployments are mostly non-overlapping.
(1) Per-Camera Tracking + Cross-Camera ReID: independent ByteTrack-style Kalman and IoU trackers per camera, then match tracklets across cameras by appearance embedding against a rolling gallery.
(2) Joint Multi-Camera Tracking (BEV): project detections or features onto a shared ground plane through calibrated homographies, fuse them into one occupancy map, and track once in world coordinates.
(3) Tracking-by-Detection + Global Association Layer: keep per-camera tracklets, then solve one hierarchical graph association over all cameras with learned edge costs combining appearance, motion, and topology priors.
(4) Chosen Hybrid: approach (3) as the backbone, borrowing (1)’s tracklet-level ReID features and switching to (2)’s geometric fusion inside the few genuinely overlapping camera clusters.

Figure 1: One cheap tracker per camera, one global association layer for the whole site, and a stored ID book so a returning object reclaims its old identity instead of getting a new one.
Clarify Before Designing:
(1) Calibration and Overlap: do we have intrinsics and extrinsics per camera, and what fraction of camera pairs actually share ground-plane coverage?
(2) Latency Mode: is this live tracking with a sub-second budget, or forensic review where a few seconds of buffering (and thus batch graph solving) is allowed?
(3) Scale and Input Size: how many cameras, at what resolution and frame rate, and can we drop to 10 fps for detection?
(4) Identity Duration: must an ID survive a 90 second walk between two non-overlapping cameras, or only a handoff across an overlapping seam?
(5) Metric and Conditions: are we scored on IDF1 or HOTA, and does the deployment include night, rain, and fleet vehicles that look identical?
Login to view more content
September 3, 2026
MSD0045 Theme Park Wait Prediction
A large theme park app (think Disneyland or Universal Studios) must show an accurate wait time for every attraction and forecast it 15 to 60 minutes ahead, so a guest can decide whether to walk across the park now or after lunch. The system ingests real-time signals: ride throughput counters, queue-length estimates from cameras or Bluetooth beacons, ride downtime events, weather, and live park attendance.
It must learn the patterns that drive wait fluctuations (time of day, day of week, season, holidays, parades and special events) and survive the non-stationarity caused by breakdowns, capacity changes, and brand-new rides. Predictions must update within seconds of a status change, and the app must show a range rather than a single number, because one estimate that is 20 minutes wrong destroys trust in every other number on the screen.
How would you design this system? Cover the real-time feature ingestion pipeline, the forecasting model architecture (sequence model vs temporal graph vs hybrid), how you handle downtime events and non-stationarity, how you produce calibrated uncertainty intervals, and how you evaluate both prediction accuracy and freshness.

The Problem: the posted number is a lagging measurement, while the guest needs a forecast that already accounts for the breakdown, the lost train, the rain, and the crowd walking over right now.
Answer
The design is a physics-anchored hybrid forecaster served from a streaming feature pipeline. A queueing-theory baseline gives the mechanical wait from live queue length and boarding throughput, and a learned model predicts the residual plus the multi-horizon trajectory. The learner is a spatiotemporal hybrid: a temporal encoder over each ride’s recent history, message passing over a park graph whose edges encode walking time and historical guest flow, and global context (attendance, weather, event calendar) injected as conditioning. Two decisions are pivotal. First, the model emits quantiles (P10, P50, P90) that are re-calibrated hourly with conformal prediction, so the app shows an interval whose stated coverage is measured, not assumed. Second, downtime is not left to the model to infer: a state machine on status events fires within seconds, switches the ride into a closure or recovery regime, and widens the band while the surge plays out.
(1) Physics Prior Plus Residual: start from $W = L/\mu$ using measured queue length and boarding rate, then learn the correction; the prior keeps the system sane when a ride has no history.
(2) Spatiotemporal Hybrid Model: per-ride temporal encoder over 4 hours of one-minute features plus graph message passing across nearby rides, because a closure spills guests onto its neighbors within minutes.
(3) Direct Multi-Horizon Quantile Heads: one forward pass emits P10/P50/P90 for the 0, 15, 30, 45, and 60-minute horizons, avoiding the error compounding of autoregressive rollout.
(4) Downtime State Machine: status events drive an explicit open / closed / recovering state that overrides the model, feeds a regime feature, and inflates the interval during the reopen surge.
(5) Streaming Features With Point-In-Time Discipline: the same transformations run in the stream (30-second online cache) and in the offline store, joined as-of prediction time so training never sees a future counter.
(6) Freshness As A First-Class SLO: every prediction carries a feature-age stamp; stale sensors degrade the system down a ladder rather than silently serving a confident stale number.

Figure 1: One batched graph inference every 30 seconds for the whole park, a seconds-latency override path for status changes, and a slow loop where realized waits measured from band scans retrain the model and recalibrate the intervals.
Clarify Before Designing:
(1) Ground Truth: how do we measure the wait a guest actually experienced (entry and boarding scan pairs from bands or the app, or only human-carried test cards), and what fraction of guests are instrumented?
(2) Scale: how many attractions per park, how many parks share the model, and how many guests open the app per day?
(3) Freshness SLO: what is the allowed lag from a status change to an updated number in the app, and how often may a displayed value be refreshed?
(4) Horizons And Display: which horizons does the product show, and is the app allowed to show a range or a range plus a point?
(5) Sensor Coverage: do all rides have queue-length sensing, or only throughput counters for some, and how are virtual queue and paid-lane guests counted?
(6) Error Asymmetry: is over-promising (say 20, actual 45) worse than under-promising, and by how much, since that ratio sets the loss weights and the displayed quantiles?
Login to view more content
September 3, 2026