Few-Shot Prompting: How Giving Examples Teaches Models Without Training

Every prompt you write is either zero-shot or few-shot. Zero-shot means you describe the task and ask the model to solve it. Few-shot means you include one or more examples of the task being done correctly, then ask the model to do the same. The distinction sounds small but the practical effect is large. Giving a model examples of what you want, in the exact format you want, is one of the most reliable ways to improve output quality without touching model weights, writing code, or paying for fine-tuning.

The paper that named the technique

The formal framing of few-shot prompting comes from 'Language Models are Few-Shot Learners' by Tom B. Brown and colleagues at OpenAI, submitted to arXiv on May 28, 2020 (arXiv:2005.14165) and presented at NeurIPS 2020. The paper introduced GPT-3, a 175-billion-parameter autoregressive language model, and tested it extensively on in-context learning: the ability to perform a new task given only a small number of examples in the prompt, with no gradient updates and no fine-tuning. Brown et al. defined three settings: zero-shot (task description only), one-shot (one example), and few-shot (two to around 100 examples, all placed in the context window). The paper found that GPT-3's few-shot performance often came close to or matched prior state-of-the-art results that had been achieved through full fine-tuning on task-specific datasets.

The few-shot capability was not engineered into GPT-3 directly. It emerged from scale. Jared Kaplan and colleagues, in a January 2020 paper (arXiv:2001.08361), showed that language model performance improves predictably as model size increases. Brown et al. demonstrated a related pattern specific to in-context learning: few-shot capability became meaningful primarily at large model scales, below which the gains from adding examples were small or negligible.

What demonstrations actually do

The intuitive explanation for why few-shot prompting works is that examples teach the model what you want. That intuition is partially right but incomplete. A 2022 paper by Sewon Min, Xinxi Lyu, Ari Holtzman, and colleagues at the University of Washington and Meta AI, titled 'Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?' (arXiv:2202.12837, published at EMNLP 2022), ran a revealing experiment: they replaced the correct labels in few-shot demonstrations with random incorrect labels, then measured how much performance dropped.

The result was surprising. Across a broad range of classification and multiple-choice tasks, tested on 12 different language models including GPT-3, randomly replacing labels barely hurt performance. The model did not need to see the correct answer in each example to benefit from having examples at all. Instead, the paper identified three things that actually drive performance: the label space (what categories or outputs are possible), the distribution of the input text (what the inputs look like), and the overall format of the sequence. Examples teach format and scope far more than they teach correctness.

Practical rules for writing few-shot prompts

• Format consistency matters more than label correctness. Choose a clear, fixed template for each example and repeat it identically. If each example has 'Input:', 'Output:', and a newline separator, use that structure for every example and for the final query.
• Three to five examples is a reasonable starting point for most tasks. Most of the gain from few-shot prompting appears in the first few examples; beyond 10 to 20, additional examples provide diminishing returns while consuming context budget.
• Select examples that cover the range of variation in your inputs. If your task involves both short and long inputs, or multiple classes, include at least one representative example of each type. A set of similar examples provides less signal than a diverse set.
• Use real examples from your actual data when possible. Examples that match the domain, vocabulary, and format of your real inputs tend to outperform generic or synthetic ones.
• If you want the model to produce a specific output format, such as JSON or a numbered list, it is more reliable to show a complete example of that format than to describe it in words.

When few-shot prompting reaches its limits

Few-shot prompting improves classification, extraction, translation, and formatting tasks substantially. It is less effective for tasks that require multi-step reasoning. Adding examples of arithmetic problems to a prompt does not reliably teach the model to solve harder arithmetic problems correctly; the model tends to jump to an answer rather than working through steps. This is the gap that chain-of-thought prompting addresses: instead of showing only inputs and outputs, chain-of-thought demonstrations include the full reasoning trace, teaching the model the steps rather than just the final answer. Few-shot prompting and chain-of-thought are frequently combined, producing few-shot chain-of-thought prompts with two to five examples that each include a worked-out reasoning chain followed by the answer.

Context length is a second constraint. Few-shot examples consume tokens. Long examples or large example sets can crowd out the actual task or exceed the model's context window. Selecting compact, representative examples matters at practical context budgets. Retrieval-augmented approaches, where examples are pulled dynamically from a database rather than hardcoded in the prompt, address this problem and connect few-shot prompting directly to RAG pipelines.

Why this technique still matters in 2026

Fine-tuning is now accessible at low cost through most major providers, and instruction-following models have strong zero-shot performance on many common tasks. Despite this, few-shot prompting remains useful for several reasons. It works immediately, without any training data pipeline or upload process. It can be adjusted in seconds during iteration. And for edge cases, specialized formats, or unusual task framings, a well-chosen set of examples still reliably outperforms a plain instruction where the model's default behavior is not quite right. PromptingIndex documents prompts that exploit few-shot structure across classification, extraction, summarization, and code generation, including patterns for keeping example sets compact while maintaining high accuracy.