Rephrase and Respond: The One-Line Prompt Fix That Fixes How LLMs Read Your Questions

Prompt quality is widely accepted as a major driver of LLM output quality. What is less appreciated is that the mismatch between how humans frame a question and how a language model internally represents it can cause failures even when the question seems perfectly clear to any human reader. 'Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves' (arXiv:2311.04205), published by Yihe Deng, Weitong Zhang, Zixiang Chen, and Quanquan Gu from UCLA in November 2023, introduces a dead-simple fix: append one instruction to any prompt and let the model rephrase the question before answering it. The technique, called RaR, improves GPT-4 accuracy across ten benchmark tasks spanning knowledge, reasoning, commonsense, and symbolic domains, with near-100% accuracy on tasks the baseline model had previously struggled to handle.

Where standard prompting silently fails

The motivating example in the paper is a question asked of GPT-4: 'Was Mother Teresa born on an even month?' GPT-4 asserts that August is an odd month. The failure is not a knowledge gap. GPT-4 knows when Mother Teresa was born. The problem is that the phrase 'even month' is genuinely ambiguous: it could mean a month with an even ordinal number (August is month 8, clearly even), or it could mean a month with an even number of days, or it could carry other interpretations. GPT-4 resolves the ambiguity the wrong way because the question was framed in a way that aligned with the human's frame but not the model's. The paper argues that this kind of frame mismatch is not an edge case. Benchmark datasets routinely contain questions that are unambiguous to human annotators but contain subtle ambiguities that cause LLMs to answer a different question than the one intended. Zero-shot chain-of-thought prompting can make this worse: on one hard knowledge task, adding 'Let's think step by step' dropped GPT-4 accuracy to 31.43%, lower than the baseline, because the model hallucinated during the intermediate reasoning steps.

The method: rephrase the question, then answer it

RaR is a single-prompt, zero-shot technique. The entire intervention is to append the following instruction to any question: 'Rephrase and expand the question, and respond.' That is the complete method. The model first restates the question in its own terms, resolving ambiguities and adding context that aligns the question with how it internally represents the relevant concepts. It then provides an answer to the rephrased version. The rephrasing step gives the model an opportunity to surface its own interpretive frame before it commits to an answer, which catches misalignments that would otherwise be invisible. The paper tests four paraphrase variants of this instruction and finds that the results are robust across all of them. Prompts like 'Reword and elaborate on the inquiry, then provide an answer' or 'Modify the original question for clarity and detail, then offer an answer' all deliver consistent improvement, so there is no magic in the exact wording.

Two-step RaR: transferring rephrasing across models

The single-prompt version of RaR works when the same model both rephrases and responds. The paper also introduces a two-step variant that separates these roles. In two-step RaR, a rephrasing LLM (the paper uses GPT-4) takes the original question and generates a clearer, more precise restatement. That rephrased question, along with the original, is then passed to a separate responding LLM. This allows a more capable model's rephrasing skill to benefit a less capable responder. The experiments show that GPT-3.5-turbo and Vicuna-13b both improve when answering questions rephrased by GPT-4, even though those models show smaller gains when doing the rephrasing themselves. The key insight is that rephrasing quality scales with model capability, so the two-step variant lets organizations trade a small amount of latency and token cost for quality transfer from a frontier model to a cheaper inference target.

Benchmark results across ten tasks

The paper evaluates RaR on GPT-4 across ten tasks covering knowledge classification, knowledge comparison, CommonsenseQA, date understanding, last letter concatenation (two-word and four-word variants), coin flip, and sports understanding, as well as Chinese idiom tasks and a bias benchmark called StereoSet. RaR improves accuracy on all ten.

• Last letter concatenation (two words): accuracy approaches 100% with RaR, up from a much lower baseline where GPT-4 frequently made symbolic reasoning errors.
• Knowledge tasks (even-day, even-month, even-year birth date classification): consistent gains across all three variants, since the rephrasing step resolves the even/odd ambiguity explicitly.
• Chinese idiom task: zero-shot CoT with 'Let's think step by step' produced 31.43% accuracy. RaR lifted accuracy to 97.73% by resolving the semantic ambiguity in the question before the model committed to a reasoning path.
• StereoSet Language Modeling Score: RaR outperforms both the original prompt and zero-shot CoT, indicating fewer bias-driven answer selections.
• CSQA (CommonsenseQA): even on a well-designed benchmark where baseline performance is already strong, RaR delivers a marginal additional gain.
• All models tested (GPT-4, GPT-3.5-turbo, Vicuna-13b) benefit from RaR, with more capable models gaining the most.

How RaR relates to chain-of-thought and when to combine them

RaR and chain-of-thought address different failure modes. Zero-shot CoT adds 'Let's think step by step' to encourage the model to produce explicit intermediate reasoning. It helps when the answer requires multi-step computation that the model would otherwise shortcut. RaR adds 'Rephrase and expand the question, and respond' to fix question interpretation before reasoning begins. It helps when the model would otherwise answer a slightly different question than the one the human intended. Because the two techniques operate at different stages of the generation process, they are complementary rather than competing. The paper shows that combining RaR with CoT produces accuracy gains beyond either technique alone on most tasks. A practical heuristic: if failures look like misinterpretation (model answers a related but wrong question), start with RaR. If failures look like incorrect arithmetic or missed steps in a chain, start with CoT. If you are unsure, try both together since the additional token cost is small compared to the quality gain.

Applying RaR in practice

The practical overhead of RaR is low. The rephrasing step adds roughly one to two sentences per prompt in input length, and the model generates a brief restatement before its answer, adding modest output tokens. No fine-tuning is needed. No few-shot examples are required. No iterative pipeline or scoring loop is involved. The method works in a single forward pass. For production use cases where question quality varies (customer support, document question answering, educational tutoring), appending 'Rephrase and expand the question, and respond' to the system prompt is a low-risk intervention worth testing before more expensive solutions. For evaluation work, the paper raises a useful caution: if a benchmark contains questions that seem clear to human annotators but are ambiguous to models, RaR accuracy and baseline accuracy may diverge significantly. That divergence is diagnostic: a large RaR gain on a benchmark task suggests the task contains question framing that the model finds ambiguous, which is useful information for dataset designers. PromptingIndex covers RaR alongside contrastive chain-of-thought, self-refine, and step-back prompting in its series on prompting strategies that alter the input stage of generation rather than the reasoning stage.