DL0199 Sensitive Data Memorization Remediation

Handle it as a privacy incident with a measurement problem at its core, not as a modeling curiosity. The first hour is containment: stop the rollout, pin and lock every checkpoint derived from the suspect run, and bring legal and privacy owners in immediately, because notification clocks (for example the 72-hour clock in GDPR Article 33) start from awareness rather than from confirmation. Then quantify before you touch anything, since “the model memorizes” is not an actionable statement until you have a per-record extraction rate under an adversarial prompt distribution, a canary exposure number, and a membership-inference AUC. Diagnosis almost always lands on the same few causes: near-duplicate records, personal data that survived scrubbing because it arrived through production logs, or too many epochs over a small sensitive shard. Only then do you pick the cheapest remediation tier that clears an agreed risk gate, escalating output guard → targeted unlearning → retrain with deduplication and DP-SGD, and you verify with adversaries you did not design the fix against.

(1) Contain Before You Explain: Freeze the rollout, revoke external checkpoint and API access, and preserve prompts, completions, and data manifests so the incident can be scoped later.
(2) Quantify With Adversarial Probes: Report extraction rate under prefix, divergence, and template attacks plus canary exposure in bits, not a handful of prompts that happened to leak.
(3) Duplication Is The Usual Root Cause: A record repeated a dozen times is far more extractable than a record seen once, so a near-duplicate audit on the flagged records comes before any algorithmic fix.
(4) Use A Cost-Ordered Remediation Ladder: Output filters buy hours, targeted unlearning buys weeks, and only retraining with deduplication and DP-SGD yields a formal per-record bound.
(5) Verification Must Be Independent: Re-measure on held-out canaries and an adaptive red team, check utility regressions, and probe for relearning, because suppressed knowledge often returns after light fine-tuning.
(6) Deletion Obligations Propagate: Embeddings, indexes, eval logs, distilled students, and downstream fine-tunes all inherit the memorization, so the remediation plan must enumerate every derived artifact.

Figure 1: The playbook, ordered by cost rather than by ambition. Each escalation step is triggered by a failed risk gate, not by preference, and the loop only closes when independent probes and a documented data lineage both hold.

The measurement phase decides everything downstream, so it deserves real engineering. Insert canaries (synthetic secrets with known formats and known insertion counts) into future runs, and for the current model estimate exposure by ranking the true secret against a candidate set of same-format alternatives under the model’s own likelihood. Complement this with black-box extraction rate: run a large, adversarially generated prompt set and count verbatim recoveries of records you know are in the training set. Two subtleties trip teams up. A single leaked record is enough to be reportable, so the metric that matters is a confidence bound on the leak probability rather than a point estimate. And zero observed hits is not zero risk: with no hits in probes, the 95% upper bound on the per-probe rate is about , which is why a 200-prompt smoke test can never clear a production gate.

Mathematical Formulation:

Where:

• is the sensitive string under test and the set of same-format candidates it is ranked against, so (30 bits for a 9-digit-scale secret) is the maximum exposure.
• is the model’s perplexity of a candidate; means the true secret is the model’s single most likely completion.
• is the empirical extraction rate, the -th adversarial prompt, the decoding procedure, and the number of probes.
• is the 95% upper confidence bound on the per-probe leak probability, which is the number a risk gate should be written against.
• is the training mechanism, and two datasets differing in one record, and the differential privacy budget that bounds any single record’s influence on the released weights.

Figure 2: Why the duplicate audit comes first. Under ordinary training, roughly 13 near-duplicates of one record are enough to push exposure past 20 bits, where a single greedy decode recovers it; mitigations such as span masking only shift the curve right, while DP-SGD flattens it at the cost of long-tail accuracy.