DL0197 Continuous Incremental Online Learning

Continuous learning is the practice of keeping a deployed model current as the serving distribution drifts, by updating its parameters from the live stream instead of freezing the artifact that passed offline evaluation. The umbrella covers three regimes that differ in how much data one update sees and how often it ships: periodic batch retraining on the accumulated corpus, incremental learning that resumes from the current weights on a new chunk or task, and online learning that takes one low-learning-rate step per example or micro-batch and never revisits it. Incremental learning still controls the update: it can mix a replay sample of old data into every batch, run a few epochs over the chunk, and gate the result behind a shadow evaluation, so forgetting is a tunable quantity. Online learning gives up that control by construction, because each example is seen once, in arrival order, with no shuffling and no held-out replay, which makes the gradient sequence strongly non-i.i.d. and makes catastrophic forgetting the default rather than the exception. The core tension in both is the stability-plasticity dilemma: enough plasticity to absorb today’s traffic, enough stability to keep yesterday’s competence.

(1) Why Forgetting Happens: gradients computed only on new data are free to move weights that encoded the old distribution, since nothing in the loss references it. Breaking the i.i.d. sampling assumption of SGD is the mechanism, not model capacity.
(2) Incremental Learning Batches The Stream: updates operate on a chunk, task, or day of data, so replay ratios, epoch counts, and evaluation gates are all design knobs.
(3) Online Learning Streams One Pass: a single step per example, bounded memory, and no revisits, which buys seconds-level freshness and pays with instability and high variance.
(4) Three Families Of Mitigation: rehearsal (replay buffers, generative replay), regularization (EWC, distillation from the previous checkpoint), and parameter isolation (adapters, LoRA branches, masks).
(5) Forgetting Is Measurable: report per-task retention and average accuracy from the accuracy matrix, never a single aggregate on fresh traffic, which hides collapse on older slices.
(6) The Dual Failure Is Loss Of Plasticity: a model updated too conservatively for months stops learning at all, so stability alone is not the objective.

Figure 1: The gradient step is identical in all three lanes. What changes is how much data one update sees, how often it ships, and what protects the old distribution: shuffling over the full corpus, an explicit replay sample plus a shadow gate, or nothing but a small learning rate and a rollback button.

In production the choice is usually driven by label latency rather than by an appetite for novelty. Recommendation and ad ranking get feedback in seconds and genuinely benefit from near-online updates, so they run streaming updates on the embedding tables and the last layers while the backbone is refreshed on a slower incremental schedule. Fraud, credit, and medical models often wait days or weeks for a trustworthy label, so a per-example update would be trained on noise; incremental daily or weekly chunks with a replay mix are the safer default. The practical recipe that survives contact with traffic is rarely exotic: keep a stratified replay buffer covering old slices, mix roughly 5 to 20 percent of every batch from it, use a small re-warmed learning rate, and gate every candidate on a fixed regression suite of historical slices before promotion. Parameter isolation is attractive when tasks are known and separable, because a per-task adapter cannot be overwritten, but it grows parameters linearly in the number of tasks and needs task identity at inference unless the routing is learned.

Mathematical Formulation:

Where:

• are the parameters after the -th update, the step size, and the arriving example; the pure online case uses each exactly once.
• is the batch actually used at step , built from fresh data and a buffer sample , with replay ratio typically in the 0.05 to 0.20 range.
• is the loss on the incoming chunk and a stability penalty anchored at the previous checkpoint , with trading plasticity for retention.
• is the diagonal Fisher information for parameter under the old task, so weights the old task depended on move less; setting reduces the penalty to plain L2-to-previous.
• is accuracy on task measured after training through task , with and indexing the chunks seen so far.
• is average accuracy over everything seen and the forgetting on task , the drop from its best-ever value to its current one.

Figure 2: Sequential updates without rehearsal do not degrade gracefully, they collapse: task 1 loses 69 accuracy points by task 10. A 5 percent replay buffer recovers most of the gap to joint retraining at a fraction of the compute, while a stability penalty alone lands in between because it also suppresses learning on the new chunk.