Design a drowsy-driving detection system that uses an in-car camera to monitor the driver and warn before they fall asleep, as offered by Volvo’s Driver Alert, Mercedes Attention Assist, and aftermarket dashcam apps. The system must run on an embedded device or a phone mounted on the dashboard, process the camera feed in real time, and detect early signs of drowsiness (eyelid closure duration, blink rate, head nodding, gaze wandering, micro-sleeps) with few false alarms so drivers do not disable it. It must work across drivers of different appearance, in varying lighting (night, tunnels, sunglasses), and it must keep the video on-device.
How would you design this system? Cover the face and landmark detection, the drowsiness signal extraction (PERCLOS, blink dynamics, head pose), the temporal model that combines signals over a window, the on-device deployment and privacy design, and how you calibrate the alert threshold to minimize false alarms without missing real drowsiness.
The Problem: the dangerous state lasts a few seconds and arrives after minutes of visible warning signs, so the system has to fire during the slow decline, at night, through sunglasses, and almost never on an alert driver.
Answer
The design is a landmark-driven feature pipeline with a learned temporal model: a cheap face detector runs on a fraction of frames with a tracker in between, a landmark network gives eye, mouth, and head-pose geometry per frame, hand-designed physiological features (PERCLOS, blink dynamics, nod and gaze statistics) are accumulated over a sliding 60-second window, and a small recurrent model maps that window to a drowsiness score. The two pivotal decisions are to keep interpretable intermediate signals rather than regressing drowsiness end-to-end from pixels, and to treat the alert threshold as a calibrated operating point on a false-alarms-per-hour axis with hysteresis, because a system that chirps at an awake driver gets unplugged on day two. Everything runs on-device on an int8 quantized graph with a near-infrared sensor, and no frame is ever written to storage or uploaded.
(1) Geometric Landmark Pipeline: face detection plus a landmark network feeding eye aspect ratio, PERCLOS, blink timing, and head pose; features are physiologically interpretable and cheap.
(2) End-to-End Video Model: a 3D CNN or video transformer over eye-region crops predicting drowsiness directly, learning subtle cues no geometric feature encodes, at a much higher data and compute cost.
(3) Hybrid Fusion (chosen): geometric features per frame plus a small GRU over the 60-second window, with per-driver normalization and hysteresis on the output score.
(4) Vehicle-Signal Baseline: steering entropy, lane-position variance, and pedal behavior with no camera at all, which is what Volvo’s original Driver Alert used; it is robust to lighting but needs minutes of degraded driving before it reacts.
Figure 1: Per-frame geometry becomes per-window physiology, and only the last stage makes a decision: pixels never leave the device and never leave RAM.
Clarify Before Designing:
(1) Sensor: RGB rolling-shutter phone camera or a dedicated near-infrared global-shutter module with an 850 nm illuminator, and at what resolution and frame rate?
(2) Compute Target: a phone with an NPU delegate, an automotive SoC, or a low-power MCU, and what sustained thermal and power budget do we get on a 4-hour drive?
(3) Alert Budget: how many false alarms per driving hour before drivers disable the feature, and are we targeting a regulated warning profile (UNECE ADDW-style) or a comfort feature?
(4) Labels: do we have annotated drowsy driving data with KSS self-reports or expert video scoring, and how many distinct drivers, or only simulator and public sets?
(5) Occlusion Requirements: must the system work through sunglasses and face masks, or may it declare “sensing unavailable” and fall back to vehicle signals?
(6) Action Space: what can the alert actually do (chime, haptic seat or wheel, escalating voice prompt, rest-stop suggestion), since the escalation ladder shapes the threshold.
Leave a Reply