Maieutic Prompting: How Recursive Explanation Trees Beat Chain-of-Thought on Commonsense Reasoning

Ask GPT-3 to explain why a statement is true and then infer its answer from that explanation, and roughly half the time the explanation will be logically disconnected from the conclusion, or the model will assign the same label to a statement and its negation, or it will flatly contradict the explanation it just generated. These are not edge cases. Jung et al. manually inspected 100 samples from a commonsense QA task using a 175B GPT-3 model and found that 41% showed a Type I failure (explanation does not logically lead to the inferred answer), 55% showed a Type II failure (model is invariant to negation, treating 'X is true' and 'X is not true' identically), and 53% showed a Type III failure (model falsifies its own generated explanation). Because many samples exhibited more than one failure type simultaneously, the overlap was substantial. The paper 'Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations,' arXiv:2205.11822 (submitted May 24, 2022; EMNLP 2022), by Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi at the University of Washington and Allen Institute for Artificial Intelligence, introduced a prompting method designed to produce correct answers even when the model's individual explanations are wrong.

The failure modes of explanation-based prompting

Standard chain-of-thought prompting asks the model to generate a single explanation and then derive a conclusion from it. The implicit assumption is that if the explanation is correct and the conclusion follows logically, the inference will be accurate. In practice, large language models violate both conditions with alarming frequency. Type I failures arise because transformer soft attention distributes probability over all tokens in the context, including tokens that are statistically prominent but logically irrelevant. The model generates an explanation that sounds plausible and then produces a conclusion that matches the surface tone of the explanation rather than its logical content. Type II failures reveal that many models treat negation as a surface token rather than a logical operator; swapping 'true' for 'not true' in a statement often has no effect on the assigned probability label. Type III failures are the most surprising: the model will generate an explanation that correctly identifies why a statement is false and then conclude that the statement is true, as if the generation of the explanation and the generation of the label are partially decoupled processes. Maieutic prompting does not attempt to fix any of these failure modes individually. Instead, it accepts that the model will generate noisy and sometimes contradictory explanations, and it builds a system that can infer the correct answer from that noisy signal.

Building the maieutic tree: abductive and recursive generation

The name comes from the Socratic maieutic method, which Vlastos (1991) described as 'a method of hypothesis elimination, steadily identifying and eliminating those that lead to contradictions.' The technique proceeds in two stages. In the first stage, the model is prompted to generate abductive explanations for both possible answers. For a true/false question Q, the model generates an explanation for why Q might be true and a separate explanation for why Q might be false. Abductive explanation means rationalizing the answer rather than deducing it: the prompt asks the model to complete 'Q is true, because...' and 'Q is false, because...' independently. Unlike standard chain-of-thought, which commits to a single explanation and then derives the conclusion, maieutic prompting remains agnostic between truth values during generation. In the second stage, the model recursively validates its own explanations. Each 1-hop explanation becomes a new question, and the same abductive prompting process runs again: 'Explanation E is true, because...' and 'Explanation E is false, because...' The recursion continues for a configurable depth, typically two to three hops. The result is a tree of propositions where each node represents a generated explanation and each edge represents a logical support relation between propositions.

Solving for consistency with weighted MAX-SAT

Once the maieutic tree is built, the inference problem is no longer a text generation task. It becomes a constraint satisfaction problem over Boolean variables. Each node in the tree is a proposition that can be assigned either True or False. The model's belief in each proposition is quantified using the language model's own generation probabilities: specifically, the ratio of probability assigned to 'True' versus 'False' completions serves as a soft confidence weight. The logical relations between propositions, whether one supports or contradicts another, are defined structurally by how the tree was generated. Abductive explanation creates a support relation: if explanation E was generated as a reason for proposition P, then E being true should be consistent with P being true. Negation relations are identified when the model generates opposing rationalizations for the same statement. The paper uses a weighted MAX-SAT solver to find the truth-value assignment across all propositions in the tree that maximizes the total weight of satisfied constraints. MAX-SAT is NP-hard in general, but the maieutic trees in the paper are small enough (typically tens of nodes) that off-the-shelf solvers handle them efficiently. The solved truth value for the original question Q is the final answer. The model's explicit generation probabilities serve as soft evidence, and the symbolic solver enforces global consistency that the language model alone cannot guarantee.

Results on three commonsense reasoning benchmarks

The paper evaluates Maieutic Prompting on three true/false commonsense reasoning benchmarks using GPT-3 (175B) as the base model. Across all three tasks, Maieutic Prompting achieves up to 20% better accuracy than state-of-the-art few-shot prompting baselines, including chain-of-thought. As a fully unsupervised approach requiring no labeled training data and no fine-tuning, Maieutic Prompting performs competitively with supervised models that were trained with task-specific labels. The gains are largest on tasks where the statements are most likely to trigger the Type II and Type III failure modes, that is, tasks with negations, subtle distinctions, or factual statements that conflict with plausible-sounding but incorrect rationalizations. The paper also evaluates robustness by perturbing both the questions and the few-shot prompts and shows that Maieutic Prompting degrades less than baseline methods under those perturbations. The interpretability benefit is explicit: because the maieutic tree and the MAX-SAT assignment are both available after inference, a user can inspect which explanations were treated as true or false and trace the chain of reasoning that produced the final answer. Code is available at github.com/jaehunjung1/Maieutic-Prompting.

• Three failure modes in standard explanation prompting: logical disconnection (41%), negation invariance (55%), self-contradiction (53%), from a 100-sample GPT-3 inspection.
• Abductive generation: the model explains why Q is true AND why Q is false before committing to either.
• Recursive validation: each explanation becomes a new question, creating a multi-hop tree of propositions.
• Weighted MAX-SAT resolves the tree into the globally most consistent set of truth-value assignments.
• Up to 20% better accuracy than chain-of-thought on commonsense true/false QA; competitive with supervised models.
• arXiv:2205.11822, submitted May 2022, EMNLP 2022; authors at University of Washington and Allen Institute for AI.

When to apply maieutic prompting and how it fits in practice

Maieutic Prompting is most valuable when the task involves binary or small-set truth judgments where logical consistency is required across multiple claims. Fact verification pipelines, commonsense plausibility scoring, and legal or medical claim evaluation are natural fits. The technique is less suited to open-ended generation tasks where there is no clear truth value to assign and where the constraint satisfaction framing does not apply. The computational overhead is higher than single-pass prompting: each question requires at least two abductive explanations per node and a recursive expansion to depth two or three, which multiplies the number of LLM calls substantially. A question with depth-2 recursion and two branches per node generates on the order of six to fourteen separate LLM calls before the solver runs. For latency-sensitive applications, this can be mitigated by batching the abductive prompts in parallel or by routing the recursive calls to a smaller model. The MAX-SAT step itself is fast once the tree is fixed. Practitioners using retrieval-augmented generation can also apply maieutic-style abductive prompting as a claim verification layer: for each retrieved passage, generate abductive explanations for why the claim is supported and why it might not be, then use the language model's own confidence to weight the evidence before aggregation. The technique generalizes the core chain-of-thought insight, that intermediate reasoning improves final accuracy, by adding two structural improvements: it reasons over multiple hypotheses simultaneously rather than committing to one, and it resolves contradictions with a principled algorithm rather than accepting whatever the model generates last. PromptingIndex covers Maieutic Prompting alongside self-consistency, reflexion, chain-of-verification, and System 2 Attention as part of its series on prompting methods that treat language model output as probabilistic evidence to be filtered and reconciled rather than as a direct answer to be trusted.