Design an image copyright violation detection system for Meta. When users upload images to Facebook or Instagram, the platform must detect whether an uploaded image infringes on a copyrighted work (a professional photograph, a movie still, a book cover, artwork) before or shortly after publication. Unlike audio or video fingerprinting, which leans on perceptual hashes for near-duplicate detection, image copyright detection must also handle crops, resizes, color adjustments, partial overlays, and collages where a copyrighted image is one component of a larger composition.
The system must embed both uploaded images and a reference database of registered works into a shared visual feature space, retrieve nearest neighbors with approximate nearest neighbor (ANN) search, and classify whether a match constitutes infringement. It must be robust to adversarial perturbations (a user adding noise or a fake watermark to evade detection), handle partial infringement where only 20 percent of the image is copyrighted, and process hundreds of millions of uploads per day at low latency. Copyright owners must be able to register their works and receive alerts when matches are found.
How would you design this system? Cover the visual embedding architecture (contrastive learning vs triplet loss vs CLIP-style vision encoder), the ANN retrieval index (FAISS vs ScaNN vs HNSW), the infringement classification strategy (similarity threshold vs learned classifier vs human review), how you handle adversarial robustness and partial infringement, and how you evaluate detection precision and recall when ground-truth labels require legal judgment.
The Problem: the same protected photograph arrives cropped, recolored, noised, and buried inside a collage, and the gate has milliseconds and no pixel-exact copy to compare against.
Answer
The design is a retrieve-then-verify matcher built on a copy-detection embedding space, not a semantic one. Every upload is hashed for exact republishes, split into a global view plus a handful of region crops, embedded by a self-supervised copy-detection descriptor trained with aggressive transform augmentation, and queried against a sharded FAISS index of registered works. Retrieved candidates go through geometric verification with local features and a calibrated classifier that scores similarity, matched area, and registry metadata together. The two pivotal decisions are to query with regions rather than one whole-image vector, which is what makes a 20 percent collage insert detectable, and to route scores into enforcement bands so only high-confidence, high-coverage matches act automatically while the ambiguous middle becomes a human review queue.
(1) Copy Descriptor Not CLIP: train a self-supervised descriptor (SSCD-style, with entropy regularization for comparable scores across images) on copy transforms; a CLIP semantic space retrieves same-topic photos and destroys precision.
(2) Multi-Region Queries: query with 1 global view plus 4 quadrants plus 4 half-scale crops, so at least one region is dominated by the protected work even in a dense collage.
(3) Two-Tier Index: a PDQ hash lookup catches pixel-near republishes in microseconds; a FAISS IVF-PQ index over roughly 400M reference vectors handles transformed copies.
(4) Retrieve Then Verify: ANN top-k feeds local-descriptor matching with RANSAC, and a learned classifier fuses inlier count, matched area fraction, descriptor score, and registration signals.
(5) Banded Enforcement: auto-allow, label plus owner alert, hold for human review, or auto-block; the top band alone acts without a human, and every action has an appeal path.
(6) Defense In Depth: adversarial augmentation during training plus three independent signal families (hash, global descriptor, local features) means an evader must break all of them at once.
Figure 1: Two paths meet at the index: uploads are embedded per region and searched, while registered works flow through ownership checks and the same encoder into the reference shards.
Clarify Before Designing:
(1) Timing And Surface: do we block before publication or detect within minutes after, and does that differ for feed posts, Stories, and paid ads?
(2) Registry Shape: how many registered works, growing how fast, and is there a premium catalog (studios, stock agencies) that deserves a stricter path than the long tail?
(3) Enforcement Policy: is the action block, label, claim for monetization, or notify-only, and who owns the appeals and counter-notice flow?
(4) Budget: uploads per day, peak QPS, allowed added latency per upload, and cost ceiling per thousand uploads?
(5) Ground Truth: can legal reviewers label a gold set, at what weekly volume, and can licensed reposts be distinguished from infringing ones in that data?
(6) Retroactivity: when a new work is registered, must we rescan historical uploads, and how far back?
Leave a Reply