DL0095 Zero-Shot vs Few-Shot

Both are inference-time conditioning: the weights never change, only the tokens placed before the query. Zero-shot prompting gives an instruction and the input, so the model must map the task description onto a behavior it already learned during pretraining or instruction tuning. Few-shot prompting (in-context learning) prepends solved demonstrations that pin down the output format, the label space, and the input distribution before the real query arrives, which is why it helps most on tasks with an unusual schema or a strict output contract. In the original GPT-3 study, TriviaQA accuracy for the 175B model moved 64.3% → 71.2% going from zero-shot to 64-shot, and the gap between the two settings shrinks as models get better instruction tuning. Few-shot prompting beats fine-tuning when labeled data is scarce (roughly tens of examples), the task spec is still changing weekly, one frozen base must serve many tasks, or no training infrastructure exists; fine-tuning wins once you have thousands of clean labels, need the lowest possible per-request cost and latency, or need behavior that no prompt reliably elicits. The decision is mostly economics plus label count, not model quality: demonstrations are paid for on every request, while a fine-tune is a one-time cost amortized over traffic.

(1) Same Weights, Different Context: neither method computes a gradient; few-shot differs from zero-shot only by the demonstration block inserted into the prompt.
(2) Demonstrations Teach Format, Not Mostly Facts: the label space, input distribution, and output template drive most of the gain, which is why even partly incorrect labels in the exemplars often still work.
(3) Cost Is Recurring: exemplars add prefill tokens per call, inflating time-to-first-token and input spend on every request forever.
(4) Fine-Tuning Trades Setup For Marginal Cost: a LoRA run costs money once and then serves a short prompt, so it wins above a traffic break-even point.
(5) Data Volume Decides The Ceiling: with a handful of labels in-context learning is usually ahead; with thousands, parameter updates reach accuracy no prompt matches.

The mechanism is worth stating precisely because it predicts the failure modes. Demonstrations act as a task locator rather than a training set: replacing gold labels with random ones from the correct label set degrades few-shot accuracy far less than removing the labels entirely, which shows the exemplars are mostly specifying which distribution to condition on. That same conditioning creates strong biases: majority-label bias (a class over-represented in the exemplars gets over-predicted), recency bias (the last exemplar dominates), and ordering sensitivity that can swing accuracy by tens of points across permutations of the same examples. Calibration on a content-free input and stratified, order-shuffled exemplar selection recover most of that variance, and any prompt tuned on a large validation set is no longer honestly “few-shot” because the selection itself consumed labels.

Figure 1: Illustrative shot-scaling behavior: most of the in-context gain arrives by and flattens after , a model fine-tuned on only 100 labels sits near the few-shot curve, and 5,000 labels put the fine-tuned small model above the frozen large model.

Mathematical Formulation:

Where:

• is the generated answer, the query, and the instruction text; the first two lines are the zero-shot and few-shot predictive distributions under identical .
• is the demonstration block and the shot count, with recovering the zero-shot case exactly.
• , , and are token lengths of the instruction, one exemplar, and the query; is the prefill length that sets time-to-first-token.
• The numeric line instantiates for exemplars of 60 tokens each, giving 1,990 prompt tokens against 70 for the zero-shot prompt.
• is the price per input token, the one-time fine-tuning cost, and the break-even request volume above which the fine-tune is cheaper; it assumes both options serve the same output length and per-token price.

Plugging in real numbers makes the trade-off concrete. At per million input tokens, the 1,920 extra tokens from 32 exemplars cost about per request, so a LoRA job pays for itself after roughly 104,000 requests. Prompt caching changes that arithmetic sharply: because the exemplar block is a fixed prefix, cached reads billed near 10% of the input rate push the break-even beyond a million requests and also cut the prefill latency penalty. That is why the honest answer to “few-shot or fine-tune” depends on traffic volume, prefix stability, and whether your serving stack caches, not on which technique sounds more advanced.

Figure 2: Illustrative cost crossover: the few-shot line grows linearly with traffic because the exemplar prefix is re-billed per call, the fine-tune is a flat one-time charge, and prompt caching flattens the few-shot slope by roughly an order of magnitude.