Chain of Density: How Iterative Entity Addition Makes LLM Summaries Match Human Writing

Summarization is one of the most common tasks people give to large language models, and one where the default outputs have a consistent failure mode: the model produces a short, readable paragraph that captures the headline but discards most of the specific information in the article. Griffin Adams, Alexander Fabbri, Faisal Ladhak, Eric Lehman, and Noemie Elhadad identified this as a density problem. A good summary should be entity-centric, meaning it should pack many specific nouns and noun phrases into a fixed number of words, matching the density of the most informative summaries human writers produce. Their paper 'From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting,' submitted to arXiv on September 8, 2023 (arXiv:2309.04269), introduces a simple iterative prompting technique called Chain of Density (CoD) that systematically closes the gap between LLM default output and human-written summary density, with a human preference study showing that the midpoint of the CoD process is what readers actually want.

The density problem in LLM summarization

When asked to summarize an article with a vanilla prompt, GPT-4 and similar models exhibit two compounding biases. The first is lead bias: the model pays disproportionate attention to the opening sentences of the source document and underweights information that appears later. A news article structured as an inverted pyramid will produce a summary that closely paraphrases the lede and ignores the supporting detail in the body. The second bias is toward sparse, readable output: the model uses hedging phrases, transitional language, and filler constructions that consume tokens without adding informational content. The result is a summary that is easy to read but informatively thin, covering one or two entities from the source where a human editor would have included six or seven. CoD addresses both problems with the same mechanism: a prompt that forces the model to confront the entities it is leaving out and add them to the next iteration, without being allowed to make the summary longer.

The CoD prompt: iterative entity injection at fixed length

The Chain of Density prompt instructs GPT-4 to generate exactly five summaries in sequence. The first summary is deliberately entity-sparse: the model is told to write a summary that is easy to read but may be missing some key information from the article. Each subsequent step follows the same structure: identify two or three entities from the article that are missing from the previous summary, then rewrite the summary to include those entities while keeping the word count identical to the previous version. Because the length cannot grow, adding new entities forces the model to compress existing content, fuse related information into denser phrases, and cut filler language. The paper defines an entity precisely: it must be relevant to the main story, specific and descriptive, novel relative to the previous summary, faithful to the article content, and drawn from anywhere in the article rather than just the opening paragraphs. The last criterion is what directly counters lead bias: the model is explicitly required to seek out entities throughout the document, not just at the top.

Human preference study: 100 articles, step 3 wins

The authors conducted a human preference study on 100 CNN DailyMail articles. Each article was summarized five times using CoD, producing a sequence of summaries from sparse to dense. Human annotators read all five summaries for each article and indicated which they preferred. The median preferred step was step 3, the midpoint of the five-step sequence. This result has a clean interpretation: steps 1 and 2 are too sparse and feel like they are leaving important information out; steps 4 and 5 are so dense that readability suffers and the summaries become hard to follow. Step 3 sits at the point where the density of specific entities closely matches the density of human-written summaries drawn from the same CNN DailyMail corpus. A GPT-4 evaluation on the same articles skewed toward higher density, preferring steps 4 and 5, which suggests that the model's own judgment of summary quality does not align with human reader preferences at high density levels.

• Adams et al. (arXiv:2309.04269, submitted September 8, 2023): five authors from Columbia University and affiliated institutions.
• CoD generates 5 summaries in a single prompt, each adding 2 to 3 new entities from the source while holding word count constant.
• Entity criteria: relevant, specific, novel relative to prior summary, faithful to the article, and drawn from anywhere in the document.
• Human preference study on 100 CNN DailyMail articles: median preferred step was step 3 of 5.
• Step 3 entity density closely matches the density of human-written summaries from the same corpus.
• CoD summaries are more abstractive, show more information fusion, and exhibit less lead bias than vanilla GPT-4 summaries.
• Dataset: 500 annotated CoD summaries and 5,000 unannotated summaries released on HuggingFace at huggingface.co/datasets/griffin/chain_of_density.

Abstraction and fusion as byproducts of compression

The paper's qualitative analysis reveals that the compression mechanism produces two structural changes in the summaries that go beyond simply adding more nouns. First, the summaries become more abstractive: instead of directly quoting or closely paraphrasing the source, the model begins to merge related facts into new phrasings not present in the original article. A phrase like 'a potential route back into the game' from the source might become 'a comeback' in the dense summary, where the compression forced the model to find a more efficient expression for the same concept. Second, the summaries show more fusion: information from two or more sentences in the source is combined into a single clause in the summary, reducing the number of separate propositions that need to be stated explicitly. Both changes are emergent consequences of the fixed-length constraint combined with the requirement to add new entities. The model is not instructed to be abstractive or to fuse information; those behaviors arise because they are the only ways to make room for the incoming entities without violating the length limit.

Applying CoD in practice

The practical recommendation from the paper is to use two to three CoD steps rather than all five when generating summaries for applications where readability matters. The full five-step sequence is useful for analysis and for understanding the tradeoff between informativeness and readability across the density spectrum, but most production summarization use cases will want to stop at step 3, where the entity density peaks in the region humans prefer. CoD is implemented as a single prompt that generates all five summaries in one call, so the extra compute cost relative to a vanilla summary prompt is modest: one inference call produces the full sequence, and the application selects the desired step from the output. The technique is model-agnostic in principle, though the original paper used GPT-4. The HuggingFace dataset at huggingface.co/datasets/griffin/chain_of_density provides 500 annotated CoD summaries with human preference labels and 5,000 unannotated summaries that can be used for fine-tuning or evaluation. PromptingIndex covers CoD alongside other structured prompting methods that use iterative or multi-step generation to improve output quality, including Self-Refine, Chain of Verification, and the Skeleton of Thought technique.