Design a multi-source notification ranking system for Meta. Facebook and Instagram send notifications from many heterogeneous sources: friend activity (someone tagged you, liked your post), group updates, event reminders, ads and promotions, and system notifications such as security alerts and policy violations. Each source has a different value to the user and a different urgency, but they all compete for the same notification slots on the user’s phone.
The system must rank notifications across all sources by predicted user value, apply frequency capping to avoid notification fatigue (sending too many notifications causes users to disable notifications entirely), deduplicate redundant notifications (three friends liking the same post should become one notification), and decide which notifications to send, suppress, or batch. Ranking must account for the user’s notification engagement history (do they open notifications, do they dismiss them), the time of day (a notification at 3 AM may annoy), and cross-source value normalization (how does a friend’s like compare to an event reminder). Decisions must be served in real time and adapt to changing user preferences.
How would you design this system? Cover the multi-source candidate generation and deduplication, the ranking model that normalizes value across heterogeneous notification types, the frequency capping and fatigue management strategy, the training label generation from user engagement signals, and how you evaluate notification ranking quality when sending fewer notifications might improve long-term engagement.
The Problem: five sources all believe their event deserves a push, the phone has a handful of slots per day, and the user is one tap away from switching the whole channel off forever.
Answer
The design is a funnel with a constrained decision at the end: per-source candidate generation → dedup and aggregation → one shared multi-task value model → a policy layer that emits send, batch, or suppress. Every source is scored by the same model into the same currency, expected long-term value of occupying one slot, so a friend tag and an event reminder become comparable numbers instead of two teams arguing about priorities. The two pivotal decisions are to make the value target include the cost of channel loss (the probability this send pushes the user toward muting or disabling notifications, multiplied by the value of all future notifications), and to implement frequency capping as an adaptive per-user send threshold rather than a fixed daily count. Hard caps and quiet hours remain as safety rails, plus a bypass lane so security and policy notifications are never traded away for engagement.
(1) Per-Source Candidate Generation: each source emits candidates with a TTL and an eligibility check (privacy, blocking, delivery capability); a candidate that expires before its slot opens is dropped rather than delayed.
(2) Dedup and Aggregation First: candidates are collapsed by an aggregation key of (object, action, time window) before ranking, so three likes on one post become one updatable card and the ranker never scores near-duplicates.
(3) Shared Multi-Task Value Model: one model with heads for open, dismiss, downstream action, and notification disable, producing calibrated probabilities that combine into a single cross-source utility.
(4) Adaptive Volume Control: a per-user threshold acts as the shadow price of a slot, tuned by a controller against a volume budget, with quiet hours, hard caps, and a bypass lane for system alerts.
(5) Batch Lane: candidates with real but sub-threshold value go into a digest that is delivered at the user’s historically responsive hour, instead of being dropped or pushed instantly.
(6) Long-Horizon Labels and Holdbacks: short-term opens train the ranker, but a permanent send-fewer holdback measures whether the policy is buying clicks today at the cost of sessions next month.
Figure 1: One scoring path for all sources: collapse duplicates, score into a shared currency, then let a budgeted policy layer decide send, batch, or suppress, with engagement feeding the next model refresh.
Clarify Before Designing:
(1) Objective Horizon: are we optimizing notification opens this week or app sessions over the next 28 days, and who owns the trade-off when they disagree?
(2) Non-Negotiable Sources: which notifications must always be delivered for legal or safety reasons (login alerts, policy enforcement), and therefore sit outside the ranking budget?
(3) Observable Feedback: which signals do the mobile platforms actually give back (delivery, open, dismiss, mute, system-level disable), and are they comparable across iOS and Android?
(4) Latency Requirement: which sources are genuinely time-critical (live video, event starting in 10 minutes) versus fine to batch by hours?
(5) Scale and Budget: how many candidates per user per day, what is the target send volume, and what infrastructure cost per candidate is acceptable?
(6) Experiment Capacity: can we hold out a permanent low-volume population long enough to measure long-term effects, and how large may it be?
Leave a Reply