Self-Refine: How Iterative Self-Feedback Improves LLM Outputs by 20% Without Extra Training

A first draft is rarely the best draft. Humans revise written work repeatedly: reading back what they wrote, identifying what is weak or unclear, and rewriting those parts until the text improves. Self-Refine, published at NeurIPS 2023, asks whether a large language model can follow the same loop. The answer from the paper (arXiv:2303.17651, submitted March 30, 2023) is yes, and the gains are measurable across a wide range of tasks. The core idea is simple: a single LLM generates an initial output, then generates feedback on that output, then uses the feedback to produce a revised output, repeating until a stopping criterion is met. No separate model is needed for the feedback step, no additional training data is required, and no reinforcement learning is involved. The same model plays all three roles: generator, critic, and refiner.

The paper and its authors

The Self-Refine paper is titled 'Self-Refine: Iterative Refinement with Self-Feedback' and was accepted at NeurIPS 2023. The 16 authors include Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. The team spans Carnegie Mellon University, Allen Institute for AI, Google DeepMind, and other institutions. Code, data, and a live demo were released at selfrefine.info at publication time. The paper evaluates Self-Refine using GPT-3.5, ChatGPT, and GPT-4 as the base models, which were the leading instruction-following LLMs available at the time of submission in March 2023.

The three-step loop: generate, feedback, refine

Self-Refine consists of three prompt-driven steps that repeat in a loop. In the GENERATE step, the model receives the task input and produces an initial output using a standard prompt. In the FEEDBACK step, the same model receives the task input and the initial output together, along with a prompt instructing it to evaluate the quality of the output and identify specific weaknesses. The feedback prompt is task-specific: for code optimization it asks about runtime efficiency, for sentiment reversal it asks whether the target sentiment has actually been achieved, and for dialogue response generation it asks about naturalness and relevance. In the REFINE step, the model receives the task input, the initial output, and the feedback, then produces a revised output. The revised output replaces the previous output, and the loop continues from the FEEDBACK step again. The stopping criterion is either a fixed number of iterations or a confidence signal in the feedback itself.

• GENERATE: standard one-shot or few-shot call producing an initial output from the task input.
• FEEDBACK: a second call to the same model, receiving the output and returning structured critique pointing to specific weaknesses.
• REFINE: a third call receiving the output plus the feedback, producing a revised version.
• Iterations continue until the stopping condition fires. The paper typically uses 3 to 4 iterations in experiments.
• All three steps use few-shot prompts constructed from 2 to 4 manually written examples per task.
• No model weights are changed, no reward model is trained, and no rollout data is collected.

The 7 tasks and what the numbers show

The paper evaluates Self-Refine on seven tasks spanning both natural language and code. The tasks are: Acronym Generation (generating a memorable acronym for a given phrase), Code Optimization (rewriting slow Python code to be faster), Code Readability Improvement (renaming variables and restructuring code for clarity), Constrained Generation (writing a sentence containing up to 30 specified concepts), Dialogue Response Generation (producing appropriate conversational replies), Math Reasoning, and Sentiment Reversal (rewriting a passage to flip its sentiment from positive to negative or vice versa).

Across all seven tasks, Self-Refine outputs were preferred by both humans and automatic metrics over single-step generation from the same model, with an average absolute improvement of approximately 20%. The largest gains appeared in the preference-based tasks. For Acronym Generation under zero-shot prompting, performance improved from 16.6% to 44.8%. For Sentiment Reversal under zero-shot prompting, performance improved from 4.4% to 71.4%. The paper also tested a version with instruction-only prompting (no few-shot examples) for Math Reasoning and found that the solve rate improved from 22.1% to 59.0%. These are improvements from the same model without any parameter changes, achieved purely through the additional prompt calls.

Why specific feedback matters

The paper includes an ablation that isolates the value of task-specific feedback versus generic feedback. When the Sentiment Reversal task was run with generic feedback prompts (asking the model to simply improve its output without task-specific guidance), performance dropped from 43.2 to 31.2. Without feedback at all, the task essentially fails. The implication is that Self-Refine does not work because the model is simply trying harder on a second attempt. It works because the feedback step surfaces specific, actionable information about what is wrong with the current output, and the refine step uses that information to make targeted corrections. Generic feedback that says 'this could be better' carries much less signal than feedback that says 'the output still contains positive language in the third sentence, which contradicts the target sentiment'.

For practitioners, this means that the feedback prompt is the most important component to get right. The task-specific feedback prompts in the Self-Refine paper are short (typically 2 to 4 few-shot examples) but carefully designed to ask precise evaluative questions about the specific quality criteria for each task. A feedback prompt that asks 'what is wrong with this code?' will underperform one that asks 'does this code complete the same computation as the original? Which lines are slower than they need to be and why?' Specificity in the feedback elicitation prompt translates directly into specificity in the critique and specificity in the correction.

Limitations and when to apply Self-Refine

Self-Refine multiplies the number of API calls per task by roughly 2 to 3 times per iteration, which increases both latency and cost. For a 3-iteration run, a single task that would require one API call now requires 7 (one generate, then three feedback-refine pairs). This makes Self-Refine most appropriate for tasks where output quality is more important than throughput and where the improvement from iteration is large enough to justify the additional calls. The paper shows that tasks with clear, measurable quality criteria and a natural notion of correctness, such as code optimization or sentiment reversal, benefit more from Self-Refine than tasks with more subjective quality criteria.

Self-Refine also assumes that the model is capable of generating useful feedback in the first place. For tasks where the model's feedback is systematically wrong or uninformative, the refinement step may not improve anything. This is a practical constraint on weaker models: a model that cannot correctly identify that its code is slow will not produce feedback that leads to a faster revision. The paper's experiments use GPT-3.5 and GPT-4, which are strong enough to generate useful task-specific feedback across the seven evaluated domains. For smaller or weaker models, the feedback quality may be insufficient to drive meaningful improvement.

PromptingIndex covers Self-Refine alongside related inference-time techniques including Chain-of-Verification (CoVe), which also uses multiple passes to catch and correct errors, and ReAct prompting, which interleaves reasoning and action in a similar iterative structure. Self-Refine occupies a distinct position: it applies the iteration to any generation task through feedback and revision, without requiring access to external tools or ground-truth verifiers, making it one of the most broadly applicable inference-time improvement techniques in the prompting literature.