Design a wake-word detection system (e.g., Amazon’s “Alexa” or Apple’s “Hey Siri”) that spots a trigger phrase in a continuous audio stream on a phone or smart speaker. It must listen always-on without draining the battery, wake almost instantly when the user speaks, and almost never fire by mistake.
How would you design this system? Cover the on-device architecture, the detection model, the training data, and how you measure accuracy.
The Problem: the detector discards nearly 100% of everything it hears, yet the one time it must fire, it has milliseconds and milliwatts to decide. Design the system that never sleeps but barely sips power.
Answer
The design is a two-stage cascade split across processors, the architecture Apple describes for its voice trigger system: a tiny streaming detector runs on the always-on low-power processor (DSP/AOP) tuned for high recall, and when it fires, a buffered audio segment goes to a larger, high-precision checker on the main application processor, optionally followed by speaker verification so only the enrolled user wakes the device. The two pivotal decisions are the cascade itself (the power budget makes one big always-on model impossible) and the operating point: false accepts are counted per unit of listening time (research reports them per hour of audio, products budget them per device-day) and they dominate the cost, so thresholds are tuned on an FA-versus-miss curve, not on accuracy.
(1) Cascade Across Processors: stage 1 is a tiny quantized DNN on the always-on processor (high recall, permissive); stage 2 is a larger checker on the application processor (high precision) fed from a ring buffer.
(2) Streaming Frontend: 16 kHz audio, 10 ms hops, log-mel filterbank frames, DNN posteriors over sub-word states, and a temporal integrator (HMM-style) that turns frame scores into one detection score.
(3) Asymmetric Objective: false accepts per device-day is the headline metric; a second-chance threshold band briefly raises sensitivity after a near-miss instead of paying FA cost all the time.
(4) Data Engine: real trigger recordings plus TTS-synthesized ones, augmented with noise, reverb, and far-field impulse responses, plus hard negatives (e.g., “seriously” for “Hey Siri”).
(5) Privacy by Construction: no audio leaves the device before activation; personalization (speaker ID) runs on-device after enrollment.
Figure 1: The cascade: a high-recall listener on the low-power chip gates a high-precision checker on the main chip; a ring buffer carries the candidate audio across so no trigger onset is lost.
Clarify Before Designing:
(1) Device Class: phone, watch, or smart speaker; battery and DSP availability decide stage sizes (a watch gets ~5% of a tiny compute budget).
(2) Trigger Spec: one phrase or several, fixed phrase or user-custom, and must it work hands-free in far-field rooms?
(3) False-Accept Budget: how many accidental activations per day are tolerable, and what do they cost (battery, privacy review, user trust)?
(4) Personalization: should only the enrolled owner’s voice trigger it, or anyone’s?
(5) Cloud Role: may a cloud verifier re-check activations, or is the device fully offline?
Leave a Reply