Few-Shot Prompting: What It Is, Examples, and How to Pick Good Examples

Suppose you're using few-shot prompting to classify support tickets. You paste in three examples and get the right labels on a handful of test inputs. That's a useful start, but it doesn't tell you whether a different set of examples would work better. The number, selection, and order of examples can change output quality. This guide compares zero-shot, one-shot, and few-shot prompting, gives copyable examples, and shows how to evaluate example choices.

What Is Few-Shot Prompting?

Few-shot prompting is a prompt engineering technique where you put a handful of input-output examples (often three to five) in the prompt. Large language models then copy their pattern, labels, or format on a new input. Nothing gets retrained: the examples only condition the model at inference time, which is why researchers call it in-context learning.

In practice, a support team that wants every ticket tagged as billing, bug, or feature_request doesn't train a classifier on thousands of labeled tickets. They paste four labeled tickets above the new one, and the model answers with one of those three labels in the same format. The four tickets define the pattern the model is asked to follow.

Zero-Shot vs. One-Shot vs. Few-Shot Prompting

The three types of prompting differ in how many solved examples the model sees before your input: zero-shot uses an instruction without demonstrations, one-shot adds one worked example, and few-shot adds several. In the GPT-3 experiments, these were three ways to specify the task in the input without updating model weights.

Here's one sentiment task written all three ways:

# Zero-shot
Label the review as positive, negative, or neutral.
Review: Setup was quick, but sync fails every night.
Label:

# One-shot: same instruction, one solved example, then the review
Review: Battery lasts all day.
Label: positive
Review: Setup was quick, but sync fails every night.
Label:

# Few-shot: same instruction, solved examples for every label, then the review
Review: Battery lasts all day.
Label: positive
Review: Arrived cracked, and support never replied.
Label: negative
Review: It's a phone case. It fits.
Label: neutral
Review: Setup was quick, but sync fails every night.
Label:

The zero-shot prompt names the allowed labels; the few-shot prompt also demonstrates a one-word answer for each. The mixed review is worth adding to your eval set to check how the model handles conflicting sentiment.

How Few-Shot Prompting Works

Brown et al. showed that GPT-3 could pick up new tasks from text demonstrations alone, with no weight updates, and called the setup few-shot learning. The examples become part of the context the model conditions on when it predicts the next tokens. Send the next request without them and the "learning" is gone, because the weights never changed.

Example count is a trade-off to test, not a fixed recipe. GPT-3's 2,048-token context window typically held 10 to 100 examples in Brown et al.'s experiments; Anthropic recommends three to five for Claude today. In the GPT-3 study, adding examples often helped but could also make results worse, so the researchers compared several values of K on a development set before testing the best one.

Few-shot prompting is one piece of context engineering: deciding what goes into the context window alongside instructions, retrieved documents, and tool results.

Few-Shot Prompting Examples

These four examples show classification, extraction, house style, and tool calls. Keep each example short and in the format you want back.

Classification

Route each support ticket to one queue: billing, bug, or feature_request.
Reply with the queue name only.

<examples>
<example>
Ticket: I was charged twice for the March invoice.
Queue: billing
</example>
<example>
Ticket: Export to CSV returns a 500 error since yesterday's update.
Queue: bug
</example>
<example>
Ticket: Could you add SAML single sign-on?
Queue: feature_request
</example>
</examples>

Ticket: The dashboard won't load in Safari, and I'd love a dark mode.
Queue:

The <example> tags separate each worked example from the instructions so Claude can distinguish the two. The last ticket is deliberately ambiguous. Decide which queue is right, then add an example like it, because ambiguous inputs are where few-shot text classification tends to break first.

Structured Extraction to JSON

Extract the job posting as JSON. Use null for anything not stated.

Input: "Senior Data Engineer, Toronto (hybrid), $150k-$170k CAD"
Output: {"title": "Senior Data Engineer", "location": "Toronto", "remote": "hybrid", "salary_min": 150000, "salary_max": 170000}

Input: "Contract ML engineer, fully remote, 6 months"
Output: {"title": "ML Engineer", "location": null, "remote": "remote", "salary_min": null, "salary_max": null}

Input: "{new_posting}"
Output:

The second example does more work than the first, since it shows what to do with missing data. If your provider offers a strict JSON schema mode, let it enforce structure and keep the examples for judgment calls like normalizing "$150k."

Matching a Format or House Style

Rewrite each change as a one-line release note. Start with a past-tense
verb, describe the user benefit, and drop ticket numbers.

Change: fix(auth): refresh token not rotated on SSO login (#4411)
Note: Fixed a bug that could sign you out early after an SSO login.

Change: feat(export): add parquet to export formats (#4502)
Note: Added Parquet as an export format for large datasets.

Change: {commit_message}
Note:

For content generation, "concise and user-focused" means different things to different models, and two few-shot examples pin it down. The same trick helps when calibrating an LLM judge, where a few scored answers with one-line rationales anchor what a 2 versus a 4 means on your rubric.

Improving Tool Calls in Agents

An agent can fail because of a malformed tool argument, like a date in the wrong format or an optional filter left out. You can put sample calls in the system prompt, and Anthropic's API also accepts an optional input_examples array on the tool definition. Each example must validate against the tool's input_schema, or the request fails with a 400 error. The Python dictionary below defines the schema and examples only. Pass it as tools=[search_orders_tool] in a configured Anthropic Messages API request; your application must still implement search_orders and return tool results.

search_orders_tool = {
"name": "search_orders",
"description": "Search a customer's orders by status and date.",
"input_schema": {
"type": "object",
"properties": {
"customer_id": {"type": "string"},
"status": {"type": "string", "enum": ["open", "shipped", "returned"]},
"since": {"type": "string", "description": "ISO 8601 date"},
},
"required": ["customer_id"],
},
"input_examples": [
{"customer_id": "C-1042", "status": "returned", "since": "2026-09-01"},
{"customer_id": "C-2210"}, # status and since are optional
],
}

How to Choose Good Few-Shot Examples

Testing GPT-3, Zhao et al. (2021) found that prompt format, example choice, and even example order can swing accuracy from near chance to near state of the art.

How Many Examples

Start with three to five and measure from there. In Min et al. (2022), model performance barely improved once k reached 8, a result to test on your own task rather than a universal example limit. Zero is a valid answer too: OpenAI's guidance for its reasoning models is to try zero-shot prompting first and add examples when the output has more complex requirements.

Diversity and Balance Across Labels

Cover every label plus the edge cases that actually show up in your traffic, and keep label counts roughly even. Zhao et al. describe a majority label bias, where predictions drift toward whichever label dominates the examples. Vary the inputs too, since four short, polite tickets teach the model nothing about the long, angry one with a stack trace pasted in.

Order and Recency Effects

In Zhao et al.'s GPT-3 experiments, recency bias meant a prompt that ends with a Negative example nudges predictions toward Negative. Lu et al. (2022) showed that a good ordering for one model doesn't transfer to another, so an order you tuned before a model upgrade isn't automatically right after it. Interleave labels rather than grouping them, and re-check when you switch models.

Clean Examples

Min et al. found that swapping gold labels for random ones barely hurt classification and multiple-choice performance. The label space, input distribution, and format were the bigger drivers. Yoo et al. (2022) pushed back in a paper titled "Ground-Truth Labels Matter," finding that the effect of correct labels varies with the setup. For extraction and generation, the output is the pattern the model copies, so a wrong example becomes a template for wrong answers. Review your pool of few-shot examples the way you'd review test fixtures.

Static vs. Dynamic Example Selection

Static prompts use the same examples for every input. Dynamic selection keeps a larger pool and picks the most relevant examples per query. Liu et al. (2021) found that retrieving semantically similar examples consistently beat random selection on their benchmarks. The flow has five steps:

1. A new query comes in.

2. Embed the query.

3. Pick the top-k most similar examples from the example pool.

4. Assemble the prompt: instructions, then the selected examples, then the query.

5. Send the prompt to the LLM.

In LangChain, SemanticSimilarityExampleSelector handles steps 2 and 3, and MaxMarginalRelevanceExampleSelector trades some similarity for diversity so you don't get k near-copies of the same ticket. To run this Python 3.10+ example, install langchain-community, langchain-core, langchain-openai, and faiss-cpu, then set OPENAI_API_KEY. It embeds a four-ticket pool and prints the two selected example dictionaries for the new ticket; selection uses paid embedding calls, but does not call a chat model or classify the ticket.

from langchain_community.vectorstores import FAISS
from langchain_core.example_selectors import SemanticSimilarityExampleSelector
from langchain_openai import OpenAIEmbeddings

examples = [
{"ticket": "I was charged twice.", "queue": "billing"},
{"ticket": "CSV export crashes.", "queue": "bug"},
{"ticket": "Please add SAML login.", "queue": "feature_request"},
{"ticket": "The invoice amount is wrong.", "queue": "billing"},
]
new_ticket = "I received a duplicate invoice."

selector = SemanticSimilarityExampleSelector.from_examples(
examples,
OpenAIEmbeddings(),
FAISS,
k=2,
input_keys=["ticket"], # embed the ticket text, not the label
)
selected = selector.select_examples({"ticket": new_ticket})
print(selected)

The cost is an embedding call per query and a prompt that changes on every request. OpenAI's prompt cache can reuse a matching prefix, so keep static instructions first and the selected examples after them. Our RFPromptManager wraps these same two LangChain selectors to pick examples per query.

Few-Shot vs. Chain-of-Thought Prompting

Few-shot examples show what the answer looks like, and chain-of-thought (CoT) prompting shows how to get there on complex tasks. Wei et al. (2022) put intermediate reasoning steps inside the examples themselves. Kojima et al. later got a zero-shot version working by adding "Let's think step by step" with no examples at all. A few-shot CoT example looks like this:

Q: A cafe sells 14 muffins in the morning and twice as many in the
afternoon. How many muffins does it sell in total?
A: Afternoon sales are 14 * 2 = 28. Total sales are 14 + 28 = 42.
#### 42

Q: A cafe sells 14 muffins in the morning and twice as many in the
afternoon. How many muffins does it sell in total?
A: Afternoon sales are 14 * 2 = 28. Total sales are 14 + 28 = 42.
#### 42

Q: A cafe sells 14 muffins in the morning and twice as many in the
afternoon. How many muffins does it sell in total?
A: Afternoon sales are 14 * 2 = 28. Total sales are 14 + 28 = 42.
#### 42

Reasoning models change the calculation. With OpenAI's reasoning models, leave out explicit chain-of-thought instructions: the models already reason internally. If you add examples, keep them consistent with the instructions. Few-shot and chain-of-thought prompting are complementary techniques, and few-shot examples still help reasoning models with output format. Just don't spend tokens teaching them how to think.

Limitations of Few-Shot Prompting

• Every example is re-sent on every call. Five 200-token examples add about 1,000 input tokens per request.

• Examples compete with retrieved documents and chat history for the same context window.

• Models can overfit to examples and copy surface features like length or phrasing. Vary your Claude examples across representative inputs and edge cases to reduce unintended patterns.

• Unbalanced or badly ordered examples skew predictions.

• There's a ceiling. If accuracy plateaus on a stable, high-volume task, look at when to fine-tune instead.

Test Few-Shot Settings Instead of Guessing

Every choice above is a setting: k, which examples, how they're selected, and where they sit. The best values depend on the model and the task. Compare a few values on held-out data and keep the winner, as Brown et al. did in the GPT-3 study.

Build an eval set of real-world inputs with known-good outputs, kept separate from your example pool. Write one scoring function (exact match for labels, field-level match for JSON), then run a zero-shot baseline next to a few variants and compare accuracy and token cost together. Don't skip the baseline. In our schema-linking benchmark of closed models, several scored better and cost less with zero-shot prompts than with few-shot prompts.

In a RAG app, few-shot examples share the context window with retrieved chunks, so test them together with your chunking and retrieval choices rather than in isolation. Our open-source experimentation engine, RapidFire AI, supports this by letting k in RFPromptManager take a List of values. The prompt manager sits in the same generator config as your retrieval spec, so k becomes one more knob in the grid alongside chunking, retrieval, and model choices. This is a configuration fragment for the linked GSM8K workflow, not a standalone experiment. Follow that tutorial's environment setup, define INSTRUCTIONS, and supply a separate examples pool with question and answer keys and at least eight rows (not the ticket/queue pool above). The Hugging Face embedding model downloads on first use and runs on CPU here. Creating the manager config alone does not run the sweep:

from rapidfireai.automl import List, RFPromptManager
from langchain_core.example_selectors import SemanticSimilarityExampleSelector
from langchain_core.prompts import PromptTemplate
from langchain_huggingface import HuggingFaceEmbeddings

prompt_manager = RFPromptManager(
instructions=INSTRUCTIONS,
examples=examples,
embedding_cfg={
"class": HuggingFaceEmbeddings,
"model_name": "sentence-transformers/all-MiniLM-L6-v2",
"model_kwargs": {"device": "cpu"},
},
example_selector_cls=SemanticSimilarityExampleSelector,
example_prompt_template=PromptTemplate(
input_variables=["question", "answer"],
template="Question: {question}\nAnswer: {answer}",
),
k=List([2, 4, 8]), # 3 values of k = 3 configs
)

To compare diversity-aware selection against plain similarity, build a second prompt manager with MaxMarginalRelevanceExampleSelector and attach it to its own generator config. Every variant runs on the same eval set, scored by your own metric function, with running estimates and confidence intervals as each shard finishes. You can stop configs that are clearly losing, then clone the strong ones and change their examples mid-run. Our GSM8K tutorial runs this kind of comparison for math word problems, with 3 versus 5 examples drawn from a pool of 10 across six configs. To search over prompt wording too, see prompt optimization.

Conclusion

Example count, selection method, and order affect outputs. Compare them on your own eval set for each model rather than adopting a rule of thumb as a fixed limit.

To try it, open the RapidFire AI RAG Colab notebook, which runs in the browser with no local setup. Or work through the GSM8K tutorial above to compare few-shot settings side by side.

Few-Shot Prompting FAQ

What Are the Three Types of Prompting in AI?

Zero-shot, one-shot, and few-shot prompting, which include no worked examples, one example, or several.

How Many Examples Should a Few-Shot Prompt Include?

For Claude, three to five examples is a useful starting point; then test fewer and more on your own eval set. For reasoning models, start at zero.

Does the Order of Examples Matter?

Yes. Zhao et al.'s GPT-3 experiments found a bias toward labels near the end of the prompt. Lu et al. found that an effective order for one model could fail on another.

Is Few-Shot Prompting the Same as Fine-Tuning?

No. Few-shot prompting changes the input and leaves the model's weights alone, so it's instant and easy to undo. Fine-tuning changes the weights, which requires training data and a training run, but can remove the need to send examples on every call.