LLMs and Generative AI β€Ί Prompt Engineering β€Ί Day 248

Day 248: Few-Shot Examples and Chain of Thought

Day 248 of 365 β€” Few-Shot Examples and Chain of Thought

Master in-context exemplar learning and step-by-step reasoning architectures: understand the theoretical mechanics of In-Context Learning (ICL) as implicit meta-gradients in attention activations, engineer balanced few-shot exemplar pools that avoid majority label and recency biases, implement Zero-Shot CoT ("Think step by step"), Manual CoT, Least-to-Most decomposition, and Self-Consistency majority voting (k >= 5 rollouts), and build an automated few-shot CoT reasoning engine with self-consistency aggregation in Python.

Course
LLMs and Generative AI
Category
Prompt Engineering
Reading time
β‰ˆ 35 min
Practical time
β‰ˆ 50 min
Lesson duration
1h 25m
Last verified
2026-08-29

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/llms-and-generative-ai/day-248-few-shot-examples-and-chain-of

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path β€” section / subsection / week / day:
    cd labs/sections/llms-and-generative-ai/day-248-few-shot-examples-and-chain-of
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work β€” read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

When you ask a standard autoregressive language model a complex mathematical, architectural, or diagnostic question in a zero-shot prompt, it faces an impossible computational bottleneck: It must jump directly to the final token in a single forward pass.

For a 70-billion-parameter Transformer, predicting the answer to β€œWhat is 347 multiplied by 829?” on token 1 forces the network to compress millions of floating-point arithmetic steps into a single forward pass across fixed residual layers. The result is almost always a convincing, confidently wrong hallucination.

By introducing Chain of Thought (CoT), you grant the model an Autoregressive Computational Scratchpad. Each intermediate token generated (β€œFirst, 300 * 800 = 240,000…”) becomes part of the key-value context window, allowing subsequent attention heads to attend to explicit partial computations.

When paired with Few-Shot In-Context Learning (ICL) and Self-Consistency Majority Voting, reasoning accuracy on complex benchmarks jumps from 35% to over 90%.

Today, you will master the theoretical mechanics of in-context meta-gradients, engineer balanced few-shot exemplar pools, implement Least-to-Most decomposition, and build an automated Self-Consistency reasoning engine in Python.


The idea in plain language

Diagram: why generating reasoning before an answer improves accuracy

Imagine asking a software engineer to architect an enterprise distributed caching cluster:


Historical background

Diagram: the historical progression of reasoning prompt techniques

In-Context Learning attention trajectory diagram showing how few-shot exemplars act as implicit meta-gradients in transformer layers

  1. 2020 (Brown et al. & GPT-3): Introduced Few-Shot In-Context Learning, demonstrating that models learn new input-output mappings from 3 to 5 examples without updating weights.
  2. 2022 (Wei et al. & Chain of Thought): Jason Wei and the Google Brain team published the seminal CoT paper, demonstrating emergent reasoning leaps on GSM8K and arithmetic benchmarks.
  3. 2022 (Kojima et al. & Zero-Shot CoT): Discovered the magic prompt trigger: β€œLet’s think step by step”, unlocking latent reasoning without requiring manual exemplars.
  4. 2023 (Wang et al. & Self-Consistency): Proved that sampling multiple stochastic reasoning paths and taking the mode/majority answer dramatically beats greedy decoding.
  5. 2023 (von Oswald et al. & Dai et al.): Proved mathematically that self-attention layers in Transformers perform an implicit meta-optimization equivalent to gradient descent during the forward pass.

What it is β€” and what it is not

What Few-Shot & CoT Reasoning IS:

What it is NOT:


Why it was created and what problems it solves

Few-Shot CoT and Self-Consistency solve 4 primary model failure modes:

  1. The Single-Pass Computation Trap: Eliminates arithmetic and logic errors by generating intermediate scratchpad tokens.
  2. Format & Style Non-Compliance: Few-shot exemplars demonstrate exact target JSON schemas and tone better than paragraphs of text instructions.
  3. Idiosyncratic Path Hallucinations: Self-Consistency prunes outlier reasoning mistakes by aggregating independent stochastic samples.
  4. Complex Multi-Hop Decomposition Failure: Breaks down 10-step problems into manageable sequential sub-questions via Least-to-Most prompting.

How it works

Let us dissect the mathematics of in-context meta-gradients, exemplar bias mitigation, and Self-Consistency trees.


1. The Theory of In-Context Learning: Implicit Gradient Descent

Diagram: how a model adapts to a task from examples in the prompt

Chain of Thought reasoning and Self-Consistency voting tree architecture comparison

How do Transformers learn from 3 examples in the prompt without backpropagation?


2. Engineering Balanced Few-Shot Exemplar Pools

Naive few-shot prompting often degrades accuracy due to 3 insidious cognitive biases (Zhao et al.):

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                   FEW-SHOT COGNITIVE BIASES & MITIGATIONS              β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Bias Name          β”‚ Manifestation           β”‚ Engineering Mitigation  β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Majority Label Biasβ”‚ Over-predicts class withβ”‚ Perfectly balance class β”‚
β”‚                    β”‚ most examples (e.g. POS)β”‚ counts (e.g. 2 POS, 2NEGβ”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Recency Bias       β”‚ Over-predicts label of  β”‚ Randomize exemplar orderβ”‚
β”‚                    β”‚ the last example in listβ”‚ or evaluate permutationsβ”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Format Bleed       β”‚ Copies exemplar text    β”‚ Use distinct synthetic  β”‚
β”‚                    β”‚ into the live answer    β”‚ domains in exemplars    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

3. The Spectrum of Chain-of-Thought Techniques

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                   CHAIN-OF-THOUGHT TAXONOMY                            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Technique          β”‚ Prompting Mechanism     β”‚ Optimal Application     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Zero-Shot CoT      β”‚ "Think step by step."   β”‚ Quick exploratory logic β”‚
β”‚ Manual Few-Shot CoTβ”‚ 3-5 Handcrafted (Q,R,A) β”‚ Structured domain tasks β”‚
β”‚ Least-to-Most CoT  β”‚ Decompose -> Solve Sub-Qβ”‚ Multi-hop graph / code  β”‚
β”‚ Self-Consistency   β”‚ Sample k paths at T=0.7 β”‚ High-stakes math / diag β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

4. Self-Consistency Majority Voting Algorithm

Diagram: how sampling multiple reasoning paths and voting improves accuracy

Instead of greedy decoding ($T = 0$), sample $k = 5$ to $k = 10$ paths at $T = 0.7$, extract the final parsed answers (a_1, a_2, ..., a_k), and compute the majority vote: a_hat = argmax sum I(a_i = a)

If $4$ out of $5$ paths conclude {"eviction_policy": "W-TinyLFU"} and $1$ concludes {"eviction_policy": "LRU"}, the system outputs W-TinyLFU with an empirical confidence score of 80%.


5. Inference Test-Time Compute Scaling Laws

Diagram: how additional inference computation substitutes for a larger model

In late 2024, AI frontier research shifted from pure pretraining parameter scaling to Inference Test-Time Compute Scaling (OpenAI o1, DeepSeek-R1):


6. Least-to-Most Decomposition for Multi-Hop Graph Reasoning

When a question requires multi-step algorithmic traversal:


7. Dynamic k-NN Exemplar Selection with Vector Embeddings

Hardcoding static few-shot examples into your codebase leads to suboptimal performance across diverse queries:


8. Faithfulness vs. Plausibility in CoT Reasoning Traces

Diagram: a plausible reasoning trace compared with a faithful one

Are LLM reasoning traces genuinely faithful reflections of the model’s internal computation?


9. Beyond Linear Chains: Tree-of-Thoughts (ToT) & Graph-of-Thoughts (GoT)

Diagram: linear reasoning compared with branching search over reasoning steps

While standard Chain-of-Thought generates a single linear trajectory:


10. Exemplar Token Budgeting & LLMLingua Prompt Compression

When combining multiple few-shot exemplars with complex CoT instructions:


An everyday analogy

Diagram: mental arithmetic compared with written working

Think of solving a complex 100-piece jigsaw puzzle:


Examples in practice

Let us build an automated Few-Shot CoT Engine with Self-Consistency Aggregation in Python:

from typing import List, Dict, Any, Callable
from collections import Counter

class FewShotCoTEngine:
    def __init__(self, task_description: str):
        self.task_description = task_description.strip()
        self.exemplars: List[Dict[str, str]] = []

    def add_exemplar(self, question: str, reasoning: str, answer: str) -> "FewShotCoTEngine":
        self.exemplars.append({
            "question": question.strip(),
            "reasoning": reasoning.strip(),
            "answer": answer.strip()
        })
        return self

    def compile_prompt(self, query: str) -> str:
        parts = [f"<instructions>\n{self.task_description}\nThink step by step before providing the final answer.\n</instructions>"]
        
        if self.exemplars:
            parts.append("<examples>")
            for i, ex in enumerate(self.exemplars):
                parts.append(
                    f'<example id="{i+1}">\n'
                    f'<input>{ex["question"]}</input>\n'
                    f'<reasoning>{ex["reasoning"]}</reasoning>\n'
                    f'<answer>{ex["answer"]}</answer>\n'
                    f'</example>'
                )
            parts.append("</examples>")
            
        parts.append(f"<query>\n{query.strip()}\n</query>\n\n<scratchpad>\nLet us reason step by step:")
        return "\n\n".join(parts)

    @staticmethod
    def aggregate_self_consistency(sampled_answers: List[str]) -> Dict[str, Any]:
        # Performs majority voting across k sampled reasoning paths
        if not sampled_answers:
            raise ValueError("Sampled answers list cannot be empty.")
            
        counts = Counter(sampled_answers)
        majority_answer, vote_count = counts.most_common(1)[0]
        confidence = vote_count / len(sampled_answers)
        
        return {
            "consensus_answer": majority_answer,
            "confidence": confidence,
            "votes": dict(counts),
            "total_samples": len(sampled_answers)
        }

Implications: security, privacy, performance, scalability, and cost

Diagram: the accuracy, cost, latency and interpretability implications of chain-of-thought prompting

  1. Token Cost of Reasoning Traces:
    • CoT increases output token volume by $3\times$ to $10\times$. For a query with 500 reasoning tokens, output cost increases proportionally.
  2. Self-Consistency Cost Multiplier:
    • Running $k = 5$ paths multiplies total inference cost by $5\times$. Reserve Self-Consistency for critical workflows (e.g. automated code deployment, medical dosage checks, financial audits).

Alternatives: free, open source, and commercial

ApproachInference LatencyAccuracy BoostBest For
Zero-Shot CoT$1\times$ Output Tokens$+15-25%$General text tasks
Manual Few-Shot CoT$+300$ Input Tokens$+30-45%$Strict domain formatting
Self-Consistency (k=5)$5\times$ Compute$+50-65%$Mission-critical logic
OpenAI o1 / o3 ReasoningInternal RL CoTState-of-the-ArtComplex math / coding

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                   REASONING PROMPT TECHNIQUE COMPARISON                β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Strategy           β”‚ Input Overhead  β”‚ Output Compute   β”‚ Sample Count β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Zero-Shot Standard β”‚ Baseline        β”‚ Minimal          β”‚ k = 1        β”‚
β”‚ Zero-Shot CoT      β”‚ +5 tokens       β”‚ Moderate         β”‚ k = 1        β”‚
β”‚ Few-Shot CoT       β”‚ +500 tokens     β”‚ Moderate         β”‚ k = 1        β”‚
β”‚ Self-Consistency   β”‚ +500 tokens     β”‚ High (k * CoT)   β”‚ k >= 5       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

When to use it β€” and when not to

Diagram: when chain-of-thought prompting helps and when it is waste

When to Use Few-Shot CoT & Self-Consistency:

When NOT to use:


The AI connection

Knowledge check

  1. Why does direct zero-shot generation fail on multi-step arithmetic and logic tasks?
  2. What is an autoregressive computational scratchpad in Transformer decoding?
  3. How does Self-Consistency majority voting filter out idiosyncratic hallucinations?
  4. What are Majority Label Bias and Recency Bias in few-shot exemplar pools?
  5. What is the mathematical formulation of In-Context Learning as implicit meta-gradients?

Hands-on exercise

In this lab, you will build and test a modular Few-Shot Chain-of-Thought compilation engine and Self-Consistency voting aggregator in Python: construct exemplar pools with XML tags, compile multi-turn CoT scratchpads, evaluate majority consensus across stochastic rollout distributions, and verify output assertions against automated test suites.

Expected output

[Few-Shot and Chain of Thought Verification Suite]
Testing Few-Shot CoT Prompt Compilation:
  Exemplar 1: Mathematical reasoning step compiled
  Exemplar 2: System architecture step compiled
  Scratchpad Trigger: '<scratchpad>\nLet us reason step by step:' [PASS]
Testing Self-Consistency Voting Aggregator:
  Sampled Rollouts (k=5): ['1081', '1081', '1051', '1081', '1081']
  Consensus Winner: '1081' | Confidence: 80.0% (4/5 votes) [VERIFIED]
  Tie-Breaking & Distribution Tracking: Validated
Test Suite: 3 passed in 0.38s

Validate your work

Run the automated test runner:

./tests/run_tests.sh

Troubleshooting

Common mistakes


Practice assignment

  1. Implement an automated Dynamic Exemplar Selector in Python that uses cosine similarity over text embeddings to retrieve the top 3 most relevant few-shot examples for any incoming user query.
  2. Build an automated CoT Step Counter that verifies the model generated at least 3 distinct deductive steps before emitting its final answer tag.

Extension challenge

Implement Tree-of-Thoughts (ToT) Search:

Quiz

Q1. What is the theoretical mechanism behind In-Context Learning (ICL) according to von Oswald et al. and Dai et al.?

  1. ICL downloads new neural weights from the internet first
  2. ICL compiles the prompt into a GPU kernel
  3. Induction heads shift activations without weight updates
  4. ICL compresses text using Huffman encoding
Show answer

Answer: C. Induction heads shift activations without weight updates

Research demonstrates that transformer self-attention implements implicit optimization during the forward pass.

Q2. Why does Chain-of-Thought (CoT) prompting drastically improve multi-step reasoning accuracy over direct zero-shot generation?

  1. Because CoT disables the GPU temperature sampling stage
  2. Because CoT makes the model type faster
  3. Because CoT reduces the prompt token count
  4. Intermediate tokens act as a computational scratchpad
Show answer

Answer: D. Intermediate tokens act as a computational scratchpad

CoT provides extra computational steps via autoregressive token generation, allowing attention heads to compute complex intermediate logic.

Q3. What is Self-Consistency Prompting (Wang et al.), and how does it improve reasoning robustness?

  1. Sample several paths and take the majority answer
  2. Asking the model to repeat its answer 3 times in the same prompt
  3. Using 5 different graphics cards simultaneously
  4. Checking if the prompt has grammatical errors
Show answer

Answer: A. Sample several paths and take the majority answer

Self-Consistency leverages ensemble majority consensus across diverse sampled reasoning paths.

Q4. What are the 3 major biases that degrade naive Few-Shot prompting?

  1. 1. Red, 2. Green, 3. Blue
  2. Majority label, recency, and common token bias
  3. 1. Audio, 2. Video, 3. Text
  4. 1. CPU, 2. GPU, 3. TPU
Show answer

Answer: B. Majority label, recency, and common token bias

Zhao et al. identified majority label bias, recency bias, and common token bias as the primary failure modes in few-shot selection.

Q5. What is Least-to-Most prompting (Zhou et al.)?

  1. A sorting algorithm for database queries
  2. A technique to minimize API token spending
  3. Break it into ordered sub-questions and chain them
  4. A method to compress image files
Show answer

Answer: C. Break it into ordered sub-questions and chain them

Least-to-Most decomposition solves complex tasks by progressively chaining solutions from simplest sub-problems to the final answer.

Glossary

In-Context Learning (ICL)
The ability of pretrained language models to perform new tasks conditioned solely on demonstration exemplars passed in the prompt context.
Chain-of-Thought (CoT)
A prompting technique that encourages the model to generate intermediate natural language reasoning steps before producing a final answer.
Self-Consistency
An ensemble decoding technique that samples multiple diverse reasoning paths at non-zero temperature and selects the most frequent final answer.
Zero-Shot CoT
Triggering step-by-step reasoning without exemplars using phrases like Let us think step by step.
Least-to-Most Prompting
Decomposing a complex problem into an ordered series of sub-problems solved sequentially in-context.
Majority Label Bias
The tendency of an LLM to over-predict the class label that appears most frequently in its few-shot examples.
Recency Bias (Exemplars)
The tendency of an LLM to predict the class label of the last few-shot exemplar in the prompt.
Computational Scratchpad
Intermediate tokens generated during autoregressive decoding that serve as working memory for subsequent attention heads.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.