AI FIELD MANUALRESEARCHERS, STUDENTS
Why asking an LLM to "write a literature review with citations" produces fabricated references, and the retrieve-then-summarize workflow that avoids it.
Last reviewed 1 September 2026
A master's student writing the literature review section of a thesis on urban heat mitigation asks ChatGPT to "summarize the key literature on cool-roof interventions with citations." The response reads like a competent literature review: named authors, plausible journal titles, specific years. Roughly a third of the citations, when checked, do not exist - the model generated statistically plausible author names and titles in the shape of a citation, not references to real papers it retrieved.
This is not a rare glitch. It's a predictable consequence of how a general-purpose chat model generates text: it produces the next most probable token, and a citation-shaped string of tokens is exactly as available to it as a real one, whether or not the underlying paper exists. Submitting this list without checking every entry is the single fastest way to damage a thesis defense.
The problem the student actually has is a workflow problem: they asked a generation tool to do a retrieval task.
Retrieve real papers first, from a tool that searches an actual indexed corpus of academic literature, before any AI-generated summarization happens. Only summarize text the tool actually retrieved and can show you the source for - never let a model produce a citation from memory.
This is the retrieve-then-generate pattern again, but for citations specifically: the retrieval step (find real papers) and the generation step (summarize what they say) must stay separated, with a verification check between them.
Tools built for literature search (Elicit, Consensus, Semantic Scholar) query an actual database of indexed papers and return real records - the paper you see is the paper that exists, because the tool is doing lookup, not free generation. A general chat model has no such database behind every claim; it is producing text that resembles a citation, which is a different operation entirely.
Verifying at the retrieval stage is far cheaper than verifying at the end: checking that a search tool's ten results are real papers (click through, confirm they exist) takes minutes; discovering that a third of a finished forty-source bibliography is fabricated after the fact means redoing the whole review.
01
Search an actual indexed corpus (Elicit, Consensus, Semantic Scholar) - real records only.
02
Click through and confirm each paper exists and matches what the tool claims.
03
Pull only findings actually present in the verified paper's text.
04
LLM drafts narrative from verified material only - explicitly no new citations.
05
Every citation in the final draft checked against the verified list before submission.
Use Elicit, Consensus, or Semantic Scholar to search the actual research question. These return real, indexed papers with links, not generated text.
Click through a sample of the results (or all of them, for a review under ~50 sources) and confirm the paper exists, matches the title and authors shown, and is actually about what the tool says it's about.
For each verified paper, pull the specific finding relevant to your review - either read the abstract/full text yourself, or use the tool's own extraction feature (Elicit's structured data extraction reads the actual paper text, it doesn't generate from memory).
Paste the verified list of papers and their extracted findings into Claude or ChatGPT and ask it to draft the narrative synthesis - explicitly instructed to reference only the papers provided and add no additional citation.
Before submitting, confirm every citation that appears in the drafted narrative is one of the papers you verified in step 2 - not a new one the model reintroduced during synthesis, which does happen even with explicit instructions not to.
Pull the final citation details (exact title, authors, year, DOI) from the original tool's record or the publisher page, not from whatever the LLM typed - transcription drift in an AI-generated bibliography is common and easy to miss.
Searching an indexed corpus of academic papers and extracting structured findings from verified sources.
elicit.com (opens in a new tab)Search engine over research papers that synthesizes findings with citations tied to real, retrieved documents.
consensus.app (opens in a new tab)Free academic search and citation graph for verifying and cross-referencing papers.
www.semanticscholar.org (opens in a new tab)Drafting the narrative synthesis from an already-verified set of papers - never the source of the citations themselves.
claude.ai (opens in a new tab)Literature-search tools have their own coverage gaps - non-English literature, very recent preprints, and paywalled or non-indexed journals may be underrepresented. A field with a lot of non-English or grey literature needs manual search to supplement this workflow, not replace it.
"Retrieved and real" is not the same as "correctly interpreted." A tool's auto-summary of a paper can still misstate a finding's strength or scope; reading the actual abstract (not just the tool's summary of it) for any source doing real work in your argument is worth the extra few minutes.
This workflow prevents fabricated citations. It does not by itself prevent cherry-picking - a search tool will return what matches your query, and a literature review that only searches for confirming evidence will find it. Deliberately search for the strongest counter-arguments too.
Advertisement
A literature review section needs ~15 sources on cool-roof urban heat mitigation interventions.
Elicit search: "cool roof reflective surface urban heat island mitigation effectiveness" returns 20 candidate papers with abstracts and links.
Each of the 20 checked by clicking through; 18 resolve to real, matching papers, 2 are dropped (one had a mismatched abstract, one link 404'd).
Elicit's extraction pulls the reported temperature-reduction finding from each of the 18 verified papers into a table.
Claude drafts a three-paragraph synthesis from the 18-paper table, instructed to cite only those papers.
Final check: all citations in the draft cross-referenced against the 18 verified records - one citation the model had subtly reworded (wrong year) was caught and corrected from the original record.
An 18-source literature review section where every citation is a real, verified paper, built in under two hours instead of the half-day a fully manual search would have taken - without the fabrication risk of asking a chat model to write it directly.
Nakoda editorial · last reviewed
This entry describes a workflow Nakoda recommends - it is not a claim about how any named tool behaves in every case, and it is not paid placement. Spotted something out of date? Tell us.
Checklist
A daily study workflow that uses AI for Socratic questioning and spaced-repetition card generation - active recall - instead of as an answer machine, plus where the line to academic dishonesty actually sits.
