Why ChatGPT gives Indian lawyers fake citations is a retrieval and verification problem, not just a prompt-writing problem. You asked ChatGPT for Supreme Court precedents on a limitation issue. It gave you three cases, each with a proper-looking case name, a year, a court, and a neat summary of the holding. You cited them in written submissions. The other side checked. None of them existed. That is not a rare edge case. It is happening often enough that the Supreme Court of India has had to take note of the problem.
The Highest Court in India Has Flagged This Problem
At this point, no advocate should treat fake AI citations as a hypothetical risk. The problem has already moved from anecdote to institutional concern. Courts are aware that AI-generated material is being used in filings, and that some of those citations do not exist in any real legal database.
That changes the practical environment immediately. Once courts are alert to the issue, judges and opposing counsel are more likely to scrutinise AI-assisted research. A fabricated citation is no longer just an embarrassing mistake. It may be read as carelessness, misrepresentation, or worse, depending on the context and the manner in which it was presented.
Professional consequences follow naturally. At the mildest end, you lose credibility before the court. At the serious end, you expose yourself to procedural consequences, reputational damage, and possible professional scrutiny. No client benefits from an advocate who delegates verification to a chatbot.
If you are still assessing which tools are even worth trusting, read AI Legal Research in India: What to Look for Before You Trust a Tool. The problem is bigger than ChatGPT. But ChatGPT is the cleanest example of why the failure happens.
For statute-heavy criminal work after 1 July 2024, citation checking also means checking whether the old IPC / CrPC label has moved under the BNS, BNSS or BSA. Use the hub guide on criminal procedure in India under BNS, BNSS and BSA and the BNS to IPC section conversion table before relying on AI-generated statutory mappings.
Why It Happens: The Technical Reason, Explained Simply
The context window problem
ChatGPT works with a context window. The simplest way to think about it is as the model's working memory for a single exchange. Whatever the model is going to answer from has to fit, directly or indirectly, into that working space.
When you ask it for legal research, the system may pull in some surrounding material, prior conversation, tool outputs, and whatever fragments it is using to answer. Indian case law is large. A single judgement can run thousands of words. Multiple judgements quickly consume available space. Once that happens, the model is no longer reasoning over a tight set of verified authorities. It is trying to produce a useful answer from partial, compressed, or poorly grounded material.
That is a structural problem for citation work. Legal research is not helped by broad, noisy context. It is helped by narrow, verified context. If the model is not operating on the exact authorities that matter, it starts leaning on pattern recognition instead.
The hallucination
When ChatGPT does not have the answer, it usually does not respond the way a careful junior would respond. A careful junior would say, "I could not trace a direct authority on that exact point." A general chatbot does something else. It tries to complete the task anyway.
That is what hallucination is in this setting. The model generates text that statistically resembles the right answer. It has seen thousands of legal citations in training, so it knows the pattern very well. It knows that Indian citations often look like a party name followed by a year, a reporter abbreviation, and a page or neutral citation.
So it can produce something like a perfect-looking Supreme Court citation even when no such matter exists. The names look plausible. The year looks plausible. The legal issue fits your question. Everything appears professionally formatted. The only problem is that the authority is invented.
Why the holding sounds accurate
This is the part that traps experienced practitioners as well. The fake case often comes with a proposition that sounds legally correct. If you ask about limitation, bail, specific performance, quashing, or maintainability, the model has absorbed the broad doctrinal language surrounding those issues. So even when the citation is false, the summary may sound exactly like something a real court would have said.
That makes the fabrication much harder to detect by instinct alone. Nothing in the answer feels absurd. There is no obvious warning sign like a ridiculous case name or a wrong statute number. The proposition may even align with the real law in the abstract. The case is still fake.
This is why fake citations are more dangerous than ordinary wrong answers. An obviously wrong answer triggers caution. A fake citation attached to a broadly accurate legal proposition invites trust.
The core design problem
ChatGPT is built to be helpful in a general sense. That means it is biased toward producing an answer. Legal research needs a different bias. It needs a system that refuses to cross the line between retrieved authority and invented support.
A tool designed for legal research should behave differently. If it cannot find an authority in its verified corpus, it should say so. If the issue is thinly covered, it should say so. If the answer depends on a narrow factual distinction, it should surface the distinction instead of smoothing it over.
That difference in design objective is the whole story. General-purpose helpfulness and legal reliability are not the same thing.
How to Spot a Fabricated Citation
A fake Indian legal citation usually does not look fake.
The format will often be right. You may see something like "Ramesh Kumar v. State of Maharashtra, (2019) 4 SCC 221" or an AIR-style reference that looks completely ordinary. The year will be plausible. The court abbreviation will be plausible. The party names will sound like a matter you have seen a hundred times.
That is why the test cannot be visual familiarity. The test is verification.
Search the full case name on Indian Kanoon. If it is a real Supreme Court or High Court judgement, it will very often appear there. If it does not, search the citation directly. If it still does not appear, search on SCC Online or Manupatra if you have access. A major authority that exists nowhere is usually not a hidden gem. It is usually an invention.
There is another warning sign as well. Ask the AI which exact paragraph contains the proposition it just attributed to the case. If it becomes vague, circular, or shifts to paraphrase instead of pointing you to the judgement, treat that as a red flag.
Before You File: A 3-Step Verification Checklist
Step 1: Search Indian Kanoon
This takes less than a minute. Search the full case name and year. For most Supreme Court and High Court authorities, that is enough to tell you whether the case exists.
Step 2: Check the holding against the actual judgement
Finding the case is only the first step. You still need to confirm that the proposition the AI attributed to it is actually borne out by the judgement. Read the relevant paragraphs yourself. Check whether you are looking at the ratio, a factual observation, or a passing comment.
Step 3: For important matters, use SCC Online or Manupatra
If you are relying on the authority in a serious matter, verify it on a paid database as well. Indian Kanoon is excellent and fast, but your filing standard should be higher than convenience. Confirm the citation form, the reporter, and the paragraph reference before you place it in a brief, petition, affidavit, or note.
This entire process takes a few minutes per citation. That is not excessive. It is the minimum cost of professional safety.
The Better Approach: Retrieval Over Generation
The alternative is not to abandon AI. The alternative is to use the right architecture.
A retrieval-based legal AI searches verified legal databases first. Only after it has found relevant authorities does it use the model to analyse, compare, and explain them. That means the model is reasoning on real materials, not improvising from memory.
That changes the risk profile fundamentally. A retrieval-based system may still miss an obscure authority. It may still give you an incomplete answer. But it is far less likely to cite a case that does not exist, because it can only work with what it actually retrieved.
This is the decisive difference between legal AI and chatbot AI. ChatGPT generates text and then gives it legal shape. A retrieval-based system starts with real authorities and then reasons from them.
The trade-off is obvious and acceptable. Retrieval-based tools may return fewer authorities on a very obscure point. But for an advocate, three verified precedents are infinitely more useful than twelve invented ones.
This is the approach Lawbot Express is built around: precision retrieval from verified Indian legal databases, with the model reasoning only on what it actually found. That is the design choice that matters most in legal research.
The Standard Is Simple: Real Citations Only
Your professional reputation and your client's matter depend on the accuracy of what you submit. AI can help you research faster, compare authorities more efficiently, and identify issues earlier. But if the system is built on generation without verification, it creates a new risk that did not exist before.
The solution is not to reject AI outright. The solution is to reject AI tools that are not built for legal research. Use systems that retrieve first, reason second, and show you enough detail to verify every authority independently.
Lawbot Express is built specifically to solve this problem. Try it free with 2 messages per day and no credit card required. And before you rely on any output, apply the same discipline described in How to Research Anticipatory Bail Precedents Using AI — A Practical Guide for Indian Advocates: verify every citation before it reaches your filing.
For source verification, use India Code for statutes and Indian Kanoon or official court websites for judgments.
Try the same verification workflow in the Lawbot Express demo, or compare research access on Lawbot Express pricing.