The PromptingIndex Blog
Guides on prompt engineering and getting the most out of AI models.
Knowledge Distillation: How Hinton, Vinyals, and Dean Taught Small Models to Learn from Soft Targets
Hinton, Vinyals, Dean (arXiv:1503.02531, 2015): temperature-scaled soft targets transfer ensemble knowledge into a single deployable model.
2026-09-08
Complexity-Based Prompting: Why Harder Examples Make Better Chain-of-Thought Prompts
Fu et al. 2022: picking CoT exemplars by reasoning step count beats manual tuning, with up to +18 accuracy on GSM8K and BigBenchHard tasks.
2026-09-08
MMLU: The 57-Subject Benchmark That Became the De Facto Standard for LLM Evaluation
Hendrycks et al. 2020: MMLU covers 57 subjects across STEM, law, and humanities. GPT-3 scored near random chance; GPT-4 hit 86.4% by 2023.
2026-09-07
Medprompt: How Microsoft Combined kNN Few-Shot, Self-Generated CoT, and Choice Shuffle Ensembling to Hit 90.2% on MedQA
Nori et al. 2023: GPT-4 with Medprompt hit 90.2% on MedQA and beat Med-PaLM 2 at 86.5% using kNN few-shot, self-generated CoT, and choice shuffle.
2026-09-07
Sparse Autoencoders and Monosemanticity: How Anthropic Decomposed 34 Million Features Inside Claude 3 Sonnet
Anthropic's 2023 to 2024 work used sparse autoencoders to extract up to 34M interpretable features from Claude 3 Sonnet, including multilingual, multimodal, and safety-relevant concepts.
2026-09-06
Thread of Thought: One Trigger Sentence That Helps LLMs Navigate Chaotic Contexts
Zhou et al. 2023: ThoT adds 'Walk me through this context in manageable parts' to fix LLM reasoning failures in RAG and multi-turn conversations.
2026-09-06
LIMA: How 1,000 Curated Examples Beat RLHF and Introduced the Superficial Alignment Hypothesis
Zhou et al. 2023: fine-tuning LLaMA-65B on 1,000 hand-curated examples beats DaVinci003 65% of the time with no RLHF required.
2026-09-05
Chain of Density: How Iterative Entity Addition Makes LLM Summaries Match Human Writing
Adams et al. 2023: CoD prompting iteratively adds entities to GPT-4 summaries without growing length, with step 3 of 5 matching human-written density.
2026-09-05
DPO: How Direct Preference Optimization Replaced Reward Models With a Classification Loss
Rafailov et al. NeurIPS 2023 Outstanding Paper Runner-up: DPO collapses the three-stage RLHF pipeline into one binary cross-entropy loss, no RL required.
2026-09-04
Buffer of Thoughts: How a Shared Thought-Template Library Cuts Reasoning Costs by 88 Percent
Yang et al. NeurIPS 2024 Spotlight: BoT stores reusable thought templates in a meta-buffer, scoring 51% higher than prior SOTA on Checkmate-in-One at 12% of the cost of tree or graph prompting.
2026-09-04
Speculative Decoding: How Draft Models Cut LLM Inference Latency Without Changing Outputs
Leviathan et al. 2022 and Chen et al. 2023: a small draft model proposes tokens the large model verifies in one pass, achieving 2x to 3x speedup with identical output distribution.
2026-09-03
Graph of Thoughts: How Arbitrary Reasoning Graphs Outperform Trees on Complex LLM Tasks
Besta et al. AAAI 2024: GoT models LLM thoughts as a graph, improving sorting quality 62% over ToT while cutting costs by more than 31%.
2026-09-03
Matryoshka Representation Learning: How a Nesting-Doll Loss Turns One Embedding Model into Many
Kusupati et al. 2022: MRL trains one embedding to be useful at 8 to 2048 dims, enabling 14x smaller storage at the same ImageNet-1K accuracy. OpenAI adopted it in text-embedding-3.
2026-09-02
Decomposed Prompting: How Modular Sub-Task Handlers Solve What Chain-of-Thought Cannot
Khot et al. ICLR 2023: DecomP decomposes complex tasks into specialized sub-task handlers, letting each module be optimized, debugged, or swapped independently.
2026-09-02
GaLore: How Projecting Gradients Instead of Weights Unlocks 7B Pretraining on a Single RTX 4090
Zhao et al. ICML 2024 showed projecting gradients into a low-rank subspace cuts optimizer memory 65.5%, enabling LLaMA 7B pretraining on one 24GB consumer GPU.
2026-09-01
Self-Consistency: Why Sampling Multiple Reasoning Paths Beats Greedy Decoding in Chain-of-Thought Prompting
Wang et al. ICLR 2023: sampling diverse chain-of-thought paths and taking majority vote adds up to 17.9 points on GSM8K over greedy decoding.
2026-09-01
AdamW: Why L2 Regularization and Weight Decay Are Not the Same in Adam
Loshchilov and Hutter's ICLR 2019 paper showed L2 and weight decay differ in Adam. AdamW decouples them and became the default optimizer for LLM training.
2026-08-31
Maieutic Prompting: How Recursive Explanation Trees Beat Chain-of-Thought on Commonsense Reasoning
Jung et al. 2022 showed that building a tree of abductive explanations and solving it with MAX-SAT yields up to 20% better accuracy than chain-of-thought prompting on commonsense QA.
2026-08-31
GPTQ: How One-Shot Quantization Fits a 175B Model onto a Single GPU
Frantar et al. 2022 showed that Hessian-based second-order quantization compresses OPT-175B to 4 bits in 4 GPU hours, with a 3.25x inference speedup and negligible accuracy loss.
2026-08-30
System 2 Attention: Teaching LLMs to Ignore What They Should Not Read
Weston and Sukhbaatar's 2023 S2A technique regenerates the input context before answering, lifting factuality from 62.8% to 80.3% and cutting sycophancy.
2026-08-30
Chinchilla: The 70B Model That Proved Bigger Is Not Always Better
DeepMind's 2022 Chinchilla paper showed that model size and training tokens must scale equally, overturning years of undertraining in LLM research.
2026-08-29
Self-Ask Prompting: How Asking Follow-Up Questions Closes the Compositionality Gap
Press et al. 2022 showed that LLMs fail to compose known facts ~40% of the time. Self-Ask fixes this by making the model ask and answer its own sub-questions first.
2026-08-29
The Switch Transformer: How Google Scaled to Trillion Parameters by Routing Each Token to One Expert
Google's 2021 Switch Transformer achieved 7x pre-training speedup over T5 by sending each token to exactly one expert, enabling trillion-parameter models.
2026-08-28
EmotionPrompt: Why Telling an LLM 'This Is Very Important to My Career' Actually Works
Microsoft Research found that appending psychological emotional stimuli to prompts boosts LLM accuracy by up to 115% on BIG-Bench tasks, no fine-tuning required.
2026-08-28
FlashAttention: The IO-Aware Algorithm That Rewrote GPU Training Speed Records
FlashAttention tiles attention in SRAM to slash HBM reads, training BERT-large 15% faster than MLPerf records with zero approximation.
2026-08-27
Rephrase and Respond: The One-Line Prompt Fix That Fixes How LLMs Read Your Questions
RaR adds a single instruction that lets LLMs restate questions in their own terms before answering, lifting accuracy across 10 benchmarks.
2026-08-27
QLoRA: Finetuning 65B Models on a Single GPU with 4-Bit Quantization
QLoRA backpropagates gradients through a frozen 4-bit model into LoRA adapters, cutting memory from 780GB to 48GB for a 65B LLaMA finetune.
2026-08-22
Contrastive Chain-of-Thought: Teaching Models What Not to Do
Contrastive CoT pairs correct reasoning with flawed examples, lifting GPT-3.5-Turbo accuracy by up to 16 points on reasoning benchmarks.
2026-08-22
ALiBi: Biasing Attention by Distance to Solve the Length Extrapolation Problem
ALiBi replaces position embeddings with a distance penalty on attention scores, letting models trained on 1024 tokens match perplexity at 2048 with 11% less memory.
2026-08-21
Active-Prompt: Using Model Uncertainty to Pick the Best Chain-of-Thought Examples
Active-Prompt queries the model multiple times, ranks questions by prediction disagreement, and annotates only the most uncertain ones as CoT exemplars.
2026-08-21
Mixture of Depths: Teaching Transformers to Skip Work They Do Not Need
Google DeepMind's MoD routes tokens around transformer blocks with a learned top-k router, matching isoFLOP baselines at up to 50% faster inference.
2026-08-20
Analogical Prompting: How LLMs Self-Generate Their Own Few-Shot Examples
Analogical prompting tells the model to recall related problems before solving a new one, eliminating manual labeling and gaining an average of +4% over standard CoT.
2026-08-20
Multi-Head Latent Attention: How DeepSeek-V2 Cut KV Cache by 93 Percent
MLA compresses KV cache into a low-rank latent vector, cutting memory by 93.3% and boosting DeepSeek-V2 generation throughput 5.76x versus DeepSeek 67B.
2026-08-19
Reflexion: Teaching Language Agents to Learn from Failure Without Fine-Tuning
Reflexion reinforces LLM agents through verbal self-reflection instead of weight updates, reaching 91% pass@1 on HumanEval and 22% gains on AlfWorld.
2026-08-19
PagedAttention: How vLLM Cut KV Cache Memory Waste from 80 Percent to 4 Percent
PagedAttention applies OS paging to LLM KV cache, reducing reserved memory waste from 60-80% to roughly 4% and lifting serving throughput up to 24x.
2026-08-18
Automatic Prompt Engineer: How LLMs Write Better Prompts Than Humans
APE uses LLMs to generate and score prompt candidates, matching human-written prompts on 21 of 24 tasks and finding a better zero-shot CoT trigger.
2026-08-18
ZeRO: How DeepSpeed Made Trillion-Parameter Training Practical
ZeRO partitions optimizer states, gradients, and weights across GPUs to cut per-device memory from 16 bytes per parameter to near zero, enabling 100B-plus model training.
2026-08-14
Plan-and-Solve Prompting: A Better Zero-Shot Alternative to Let's Think Step by Step
PS and PS+ prompting replace 'Let's think step by step' with structured planning instructions, outperforming Zero-Shot-CoT across ten benchmark datasets.
2026-08-14
SwiGLU: The Activation Function Inside LLaMA, PaLM, Mistral, and Gemma
How Noam Shazeer's 2020 paper on GLU variants replaced ReLU in every major LLM, using three weight matrices and 2/3 the standard hidden dimension.
2026-08-13
Auto-CoT: Eliminating Manual Work from Chain-of-Thought Prompting
Auto-CoT uses clustering and Zero-Shot-CoT to build diverse reasoning demonstrations automatically, matching hand-crafted CoT across ten benchmarks with GPT-3.
2026-08-13
BitNet b1.58: How Ternary Weights Cut LLM Memory by 82% Without Losing Accuracy
BitNet b1.58 (arXiv:2402.17764) stores every weight as -1, 0, or 1, matching FP16 quality while slashing energy use by up to 82% and running a 100B model on one CPU.
2026-08-12
Generated Knowledge Prompting: Ask the Model First, Then Answer
Generated Knowledge Prompting asks a language model to produce relevant facts before answering, gaining 7 to 10% accuracy on commonsense benchmarks.
2026-08-12
Mamba: How Selective State Spaces Achieve 5x Faster Inference Than Transformers
Mamba (arXiv:2312.00752) by Gu and Dao introduced selective state spaces, achieving linear sequence scaling and 5x Transformer throughput.
2026-08-11
OPRO: How Google DeepMind Taught LLMs to Write Their Own Prompts
OPRO (arXiv:2309.03409, ICLR 2024) lets an LLM act as its own optimizer, finding prompts that beat human-written ones by up to 8 points on GSM8K and 50 points on Big-Bench Hard.
2026-08-11
Rotary Position Embedding: How RoPE Powers Llama, Mistral, and Nearly Every Modern Open LLM
RoPE (arXiv:2104.09864, 2021) encodes position as complex-number rotations on query and key vectors, making attention scores depend on relative rather than absolute position.
2026-08-10
Self-Refine: How Iterative Self-Feedback Improves LLM Outputs by 20% Without Extra Training
Self-Refine (NeurIPS 2023) prompts one LLM to generate, critique, and revise its own output in a loop, gaining roughly 20% on 7 diverse tasks with no added training.
2026-08-10
Grouped Query Attention: How GQA Cuts KV Cache Size Without Sacrificing Quality
GQA from Google Research shares KV heads across query groups, powering Llama 2, Llama 3, and Mistral 7B with full-attention quality at near-MQA inference speed.
2026-08-07
Skeleton-of-Thought: How Parallel Prompting Cuts LLM Latency Up to 2.39x
SoT from Tsinghua University guides LLMs to outline an answer first, then expand each point in parallel, cutting end-to-end latency without changing model weights.
2026-08-07
GRPO: How DeepSeek Replaced PPO's Critic With a Group of Answers
Group Relative Policy Optimization cuts a full critic model from RL training by scoring each response against its peers, not against a learned value function.
2026-08-06
Chain-of-Verification: How Meta AI Taught Models to Fact-Check Their Own Outputs
CoVe from Meta AI cuts hallucinations by having a model generate verification questions about its draft, answer them independently, then revise its response.
2026-08-06
DPO: How Stanford Researchers Replaced RL Training With a Single Classification Loss
Direct Preference Optimization eliminates reward model training and PPO by showing the optimal RLHF policy is solvable in closed form from preference data alone.
2026-08-05
Step-Back Prompting: Ask the Abstract Question First, Then Answer the Specific One
Step-Back Prompting from Google DeepMind boosts PaLM-2L on MMLU Physics by 7% and TimeQA by 27% by having models derive principles before solving details.
2026-08-05
Constitutional AI: How Anthropic Trained a Harmless Assistant Without Human Labels for Harm
Constitutional AI replaces human harm labels with AI self-critique and RLAIF, producing an assistant that is preferred over RLHF-trained models by crowdworkers.
2026-08-04
PAL: How Writing Code Instead of Prose Makes Language Models Better at Math
PAL replaces chain-of-thought arithmetic with Python code the interpreter runs, letting Codex outperform PaLM-540B on math word problems by 15 points.
2026-08-04
LoRA: How Low-Rank Adaptation Makes Fine-Tuning Large Language Models Practical
LoRA cuts trainable parameters by 10,000 times using rank decomposition, letting practitioners fine-tune billion-parameter models on a single GPU.
2026-07-18
Least-to-Most Prompting: How Breaking Problems Into Subproblems Unlocks Harder Reasoning
Least-to-most prompting beats chain-of-thought on hard compositional tasks by decomposing each problem into simpler subproblems solved in sequence.
2026-07-18
The KV Cache: How Language Models Remember What They Have Already Read
The KV cache lets LLMs skip recomputing prior tokens on every generation step, but it can consume tens of gigabytes at long context lengths.
2026-07-16
RAG: How Retrieval-Augmented Generation Lets Language Models Answer from Your Documents
RAG gives language models access to external knowledge at query time by retrieving relevant passages and injecting them as context.
2026-07-16
RLHF: The Three-Stage Training Pipeline That Made Language Models Follow Instructions
The 2022 InstructGPT paper introduced RLHF, producing a 1.3B model preferred over 175B GPT-3, via supervised fine-tuning, reward modeling, and PPO.
2026-07-15
ReAct Prompting: Combining Reasoning and Acting to Ground LLMs in the Real World
ReAct interleaves reasoning traces with external actions, letting models search, verify facts, and navigate tasks using just one or two in-context examples.
2026-07-15
Mixture of Experts: How Modern LLMs Activate Only a Fraction of Their Parameters
Sparse Mixture of Experts lets language models hold hundreds of billions of parameters while activating only a fraction per token at inference time.
2026-07-14
Tree of Thoughts: How Branching Reasoning Paths Solve Problems Chain-of-Thought Cannot
Tree of Thoughts lets a model explore and evaluate multiple reasoning branches, lifting GPT-4's Game of 24 solve rate from 4 percent to 74 percent.
2026-07-14
Chinchilla Scaling Laws: The 2022 Paper That Showed GPT-3 Was Undertrained
DeepMind's 2022 Chinchilla paper proved GPT-3 was undertrained and that model size and token count must scale equally for compute-optimal training.
2026-07-13
Speculative Decoding: How LLMs Generate 2x to 3x More Tokens Without Changing Their Weights
Speculative decoding speeds up LLM inference 2x to 3x using a smaller draft model to guess tokens the large model then verifies in parallel.
2026-07-12
Few-Shot Prompting: How Giving Examples Teaches Models Without Training
Few-shot prompting adds worked examples to your prompt so a model can match format and scope with no training, no fine-tuning, and no code changes.
2026-07-12
FlashAttention: The Algorithm That Made Long-Context AI Possible
How Tri Dao's FlashAttention turned attention's quadratic memory cost into a linear one, enabling context windows to grow from 4K to over 1M tokens.
2026-07-11
Self-Consistency Prompting: How Majority Voting Over Reasoning Paths Gets Better Answers
Self-consistency runs a prompt multiple times and picks the most common answer, cutting errors on math and logic tasks by double digits.
2026-07-11
Thinking Longer: How Inference-Time Compute Reshaped AI Reasoning
How test-time compute turned AI reasoning inside out: why letting models think longer beats training larger ones, and what it costs to do it.
2026-07-10
Chain-of-Thought Prompting: Make the Model Show Its Work
Chain-of-thought prompting boosts AI reasoning on hard problems by asking for step-by-step thinking. Here is how it works and when to use it.
2026-07-10
Tokens, Not Words: How LLMs Actually Read Your Prompt
Language models never see letters or words. They see tokens. Here is how tokenization works and why it quietly shapes cost, context, and accuracy.
2026-07-07
Quantization: How 70-Billion-Parameter Models Fit on One GPU
How quantization slashes AI model memory by reducing numerical precision, turning a 140 GB download into something a single GPU can handle.
2026-07-06
Stop Regenerating: Fix Your Prompts Instead of Replacing Them
Clicking regenerate is hoping a bad prompt magically improves. Here is how to diagnose what went wrong and fix it with one surgical addition.
2026-07-06
Mixture of Experts: How Huge AI Models Stay Fast
How mixture-of-experts models like Mixtral pack huge parameter counts while staying cheap to run, by activating only a few expert networks per token.
2026-07-05
The Leading Companies in AI (2026): Who Builds the Models You Prompt
A field guide to the companies leading AI in 2026, from frontier labs like OpenAI, Anthropic, and Google DeepMind to the specialists in search, image, voice, and open models.
2026-07-04
How Large Language Models Actually Learn: Pretraining, Fine-Tuning, and RLHF
A plain-English deep dive into how LLMs like GPT, Claude, and Gemini are built, from next-token pretraining to instruction tuning and RLHF.
2026-07-03
The Best AI Prompts for Productivity
Practical AI prompts to plan your day, clear your inbox, summarize long documents, and turn messy notes into action items.
2026-07-03
Prompt Engineering Basics: The Three Levers That Matter
Prompt engineering demystified. Learn the three levers (specificity, examples, and iteration) that separate mediocre prompts from great ones.
2026-07-03
How to Write a Great AI Prompt
A simple framework for writing prompts that get better results from ChatGPT, Claude, and other AI models: role, task, context, format, and constraints.
2026-07-03