Design the copyright detection system for a video platform like YouTube. Every uploaded video (hundreds of hours per minute) must be scanned against a reference corpus of tens of millions of copyrighted songs, movies, and clips, and matched even when the content is pitch-shifted, cropped, overlaid with voiceover, recorded off a screen, or remixed. The system must return matches within minutes so that rightsholders can block, monetize, or track the use, and it must keep the false-positive rate low enough that legitimate fair use and original content are not wrongly claimed.
How would you design this system? Cover the audio and video fingerprinting representations, the matching index at scale, the latency and throughput budget for the upload firehose, how you handle partial and transformed matches, and how disputes and appeals are routed.
The Problem: the copy is never bit-identical, the corpus is tens of millions of references, the answer is owed in minutes, and a wrong claim silences a legitimate creator.
Answer
The design is a two-stage fingerprint retrieval system: cheap, high-recall candidate generation from compact per-modality fingerprints, then strict temporal verification that only accepts a candidate if many hash hits agree on a single time offset. Audio and video are fingerprinted separately because their attack surfaces differ. Audio uses spectral peak landmark hashes plus a neural fallback for pitch and tempo edits, while video uses learned copy-detection embeddings that survive cropping, overlays, and screen recording. The two pivotal decisions are to make the verification stage, not the recall stage, own precision, and to convert model confidence into tiered actions (auto-claim, human review, track-only) so that an uncertain match costs a reviewer’s time rather than a creator’s channel.
(1) Dual-Modality Fingerprints: audio landmark hashes of the form 
at roughly 30 hashes per second, and video frame descriptors sampled at 2 to 4 fps after letterbox and border normalization.
(2) Two-Stage Retrieval: a sharded inverted index (plus ANN search over neural embeddings) proposes a few hundred candidate references per query chunk; verification rejects almost all of them.
(3) Offset Voting And Segment Stitching: a real match produces a spike in the histogram of 
; adjacent winning chunks are stitched into segments with a minimum matched duration.
(4) Chunked Parallel Ingest: uploads are split into 10-second chunks that fingerprint in parallel, with a priority queue so premieres and large channels finish first.
(5) Tiered Actions: high confidence auto-claims, medium confidence routes to human review or track-only, low confidence is logged and dropped.
(6) Dispute Loop As Training Data: every appeal outcome becomes a label, feeding whitelists, threshold retuning, and reference-corpus cleanup.
Figure 1: One path for uploads (fingerprint → lookup → verify → policy) and one offline path that fingerprints rightsholder references with the same extractor; disputes flow back from the policy engine into verification thresholds and whitelists.
Clarify Before Designing:
(1) Scope Of Rights: do we match sound recordings only, or also compositions and melodies, since a cover song copies the composition without copying any audio?
(2) Verdict SLA: must the scan block publishing, or may the video go live and be claimed minutes later, and does the SLA differ for monetized channels?
(3) Corpus Scale And Registration: how many references, how fast is the corpus growing, and who is allowed to register a reference?
(4) Cost Asymmetry: what false-claim rate is acceptable versus missed detections, and who pays for human review of the uncertain band?
(5) Transformation Catalog: which edits must be caught (crop, pitch shift, speed change, voiceover, screen capture, remix, AI covers), because that choice decides the fingerprint representation?
(6) Available Actions: block, monetize, or track, and are the policies territory-specific?
Leave a Reply