The PromptingIndex Blog
Guides on prompt engineering and getting the most out of AI models.
LoRA: How Low-Rank Adaptation Makes Fine-Tuning Large Language Models Practical
LoRA cuts trainable parameters by 10,000 times using rank decomposition, letting practitioners fine-tune billion-parameter models on a single GPU.
2026-07-18
Least-to-Most Prompting: How Breaking Problems Into Subproblems Unlocks Harder Reasoning
Least-to-most prompting beats chain-of-thought on hard compositional tasks by decomposing each problem into simpler subproblems solved in sequence.
2026-07-18
The KV Cache: How Language Models Remember What They Have Already Read
The KV cache lets LLMs skip recomputing prior tokens on every generation step, but it can consume tens of gigabytes at long context lengths.
2026-07-16
RAG: How Retrieval-Augmented Generation Lets Language Models Answer from Your Documents
RAG gives language models access to external knowledge at query time by retrieving relevant passages and injecting them as context.
2026-07-16
RLHF: The Three-Stage Training Pipeline That Made Language Models Follow Instructions
The 2022 InstructGPT paper introduced RLHF, producing a 1.3B model preferred over 175B GPT-3, via supervised fine-tuning, reward modeling, and PPO.
2026-07-15
ReAct Prompting: Combining Reasoning and Acting to Ground LLMs in the Real World
ReAct interleaves reasoning traces with external actions, letting models search, verify facts, and navigate tasks using just one or two in-context examples.
2026-07-15
Mixture of Experts: How Modern LLMs Activate Only a Fraction of Their Parameters
Sparse Mixture of Experts lets language models hold hundreds of billions of parameters while activating only a fraction per token at inference time.
2026-07-14
Tree of Thoughts: How Branching Reasoning Paths Solve Problems Chain-of-Thought Cannot
Tree of Thoughts lets a model explore and evaluate multiple reasoning branches, lifting GPT-4's Game of 24 solve rate from 4 percent to 74 percent.
2026-07-14
Chinchilla Scaling Laws: The 2022 Paper That Showed GPT-3 Was Undertrained
DeepMind's 2022 Chinchilla paper proved GPT-3 was undertrained and that model size and token count must scale equally for compute-optimal training.
2026-07-13
Speculative Decoding: How LLMs Generate 2x to 3x More Tokens Without Changing Their Weights
Speculative decoding speeds up LLM inference 2x to 3x using a smaller draft model to guess tokens the large model then verifies in parallel.
2026-07-12
Few-Shot Prompting: How Giving Examples Teaches Models Without Training
Few-shot prompting adds worked examples to your prompt so a model can match format and scope with no training, no fine-tuning, and no code changes.
2026-07-12
FlashAttention: The Algorithm That Made Long-Context AI Possible
How Tri Dao's FlashAttention turned attention's quadratic memory cost into a linear one, enabling context windows to grow from 4K to over 1M tokens.
2026-07-11
Self-Consistency Prompting: How Majority Voting Over Reasoning Paths Gets Better Answers
Self-consistency runs a prompt multiple times and picks the most common answer, cutting errors on math and logic tasks by double digits.
2026-07-11
Thinking Longer: How Inference-Time Compute Reshaped AI Reasoning
How test-time compute turned AI reasoning inside out: why letting models think longer beats training larger ones, and what it costs to do it.
2026-07-10
Chain-of-Thought Prompting: Make the Model Show Its Work
Chain-of-thought prompting boosts AI reasoning on hard problems by asking for step-by-step thinking. Here is how it works and when to use it.
2026-07-10
Tokens, Not Words: How LLMs Actually Read Your Prompt
Language models never see letters or words. They see tokens. Here is how tokenization works and why it quietly shapes cost, context, and accuracy.
2026-07-07
Quantization: How 70-Billion-Parameter Models Fit on One GPU
How quantization slashes AI model memory by reducing numerical precision, turning a 140 GB download into something a single GPU can handle.
2026-07-06
Stop Regenerating: Fix Your Prompts Instead of Replacing Them
Clicking regenerate is hoping a bad prompt magically improves. Here is how to diagnose what went wrong and fix it with one surgical addition.
2026-07-06
Mixture of Experts: How Huge AI Models Stay Fast
How mixture-of-experts models like Mixtral pack huge parameter counts while staying cheap to run, by activating only a few expert networks per token.
2026-07-05
The Leading Companies in AI (2026): Who Builds the Models You Prompt
A field guide to the companies leading AI in 2026, from frontier labs like OpenAI, Anthropic, and Google DeepMind to the specialists in search, image, voice, and open models.
2026-07-04
How Large Language Models Actually Learn: Pretraining, Fine-Tuning, and RLHF
A plain-English deep dive into how LLMs like GPT, Claude, and Gemini are built, from next-token pretraining to instruction tuning and RLHF.
2026-07-03
The Best AI Prompts for Productivity
Practical AI prompts to plan your day, clear your inbox, summarize long documents, and turn messy notes into action items.
2026-07-03
Prompt Engineering Basics: The Three Levers That Matter
Prompt engineering demystified. Learn the three levers (specificity, examples, and iteration) that separate mediocre prompts from great ones.
2026-07-03
How to Write a Great AI Prompt
A simple framework for writing prompts that get better results from ChatGPT, Claude, and other AI models: role, task, context, format, and constraints.
2026-07-03