AI-hallucinated case law: how fake citations are getting lawyers sanctioned in India (and how to avoid it)

What is AI-hallucinated case law?

AI-hallucinated case law is exactly what it sounds like: case citations, judgment names, quoted passages or paragraph numbers that a generative AI tool produces confidently but that are false. The case may not exist at all. Or it may be a real case with an invented holding, a wrong citation, or a paragraph that was never in it. The word “hallucination” is the industry’s own term for output that is fluent, plausible and simply untrue.

What makes legal hallucinations so dangerous is that they are indistinguishable from the real thing on their face. Ask a general-purpose chatbot for Indian authorities on, say, the limitation period for an insolvency application, and it may hand you a neatly formatted list: a party name that sounds right, a citation in the correct “(YEAR) VOL SCC PAGE” shape, a confident one-line ratio, and a paragraph reference. Everything looks like a lawyer wrote it. Nothing about the format tells you the case was born in a language model three seconds ago.

Would you catch it on a busy day? That’s the trap. A fabricated citation does not come with a warning label. It reads like authority, it slots into a written submission without friction, and unless you actually open the report and read the judgment, you have no way of knowing it is fiction. The Supreme Court captured this quality vividly in the 2 July 2026 ruling, describing fake material entering the justice system as invisible and insidious, seeping into a decision without anyone noticing until the damage is done.

There is an important distinction to draw here, because not every AI mistake is a hallucination in this narrow sense. Broadly, the fabrications fall into three buckets. First, the wholly invented case: a party name and citation that correspond to no judgment anywhere. Second, the real case, wrong content: a genuine citation with an AI-written holding or paragraph that the actual judgment never contained. Third, the misattributed quote: a real principle from Case A confidently pinned to Case B. All three have surfaced in Indian matters. All three are equally fatal to a filing.

Why AI invents citations that look completely real

To avoid the problem, you have to understand its root, and the root is architectural. Tools like ChatGPT and Claude are large language models. At their core, they predict the most probable next word given everything before it. They are extraordinary at language: summarising, rephrasing, structuring, drafting, translating. But they are not databases, and they were never built to be. When you ask one to “cite Indian cases on default bail,” it does not look anything up. It generates the most statistically likely sequence of words that resembles a case citation. Sometimes that sequence happens to match a real case. Often it does not.

Think of it this way. If you asked a very well-read person to recall, off the top of their head, the exact citation and paragraph number of a judgment they skimmed years ago, they might produce something that feels right but is subtly wrong. An LLM does that at scale, with far more confidence and no memory of ever having been unsure. It has no internal flag that says “I am now guessing.” Every answer arrives in the same assured tone, whether it is rock-solid or entirely fabricated.

Two structural limits make this worse for Indian legal work specifically. The first is the knowledge cutoff. Every model is trained on data up to a fixed point, and it knows nothing issued after that date: no notification, no amendment, no judgment. The second, and the one that ends careers, is that these general-purpose models do not reliably know Indian case law from memory in the first place. Indian judgments are underrepresented in their training data compared with the vast volume of American and English material, so when pushed for Indian authorities without any source to work from, the model fills the gap with confident invention.

This is precisely where India’s home-grown legal AI tools differ, and the difference is worth understanding. Products such as Manupatra’s AI assistant, SCC Online’s research tools, Indian Kanoon’s offering and CaseMine’s AMICUS run what is called retrieval-augmented generation: they answer by first searching a real, curated corpus of Indian judgments, then summarising what they actually found. That grounding sharply reduces hallucination, though it does not eliminate it entirely. General-purpose ChatGPT and Claude have no such corpus unless you supply one. And that single fact, the presence or absence of a real source, separates safe AI use from dangerous AI use.

The Indian cases: a growing wall of fake-citation sanctions

Is this a real risk in India or an imported American scare story? Look at the record. The cautionary cases are no longer hypothetical, and they now span tax tribunals, High Courts and the Supreme Court itself.

The one that turned heads in the tax bar was the Buckeye Trust matter. In an order dated 30 December 2024, the Bengaluru bench of the Income Tax Appellate Tribunal, deciding a trust-taxation dispute running into roughly Rs 669 crore, cited three Supreme Court judgments and one Madras High Court ruling. None of them existed. When the fabrications came to light, the Tribunal had to recall its own order under Section 254(2) of the Income-tax Act, 1961, the provision that lets it rectify a mistake apparent from the record. An order of that magnitude, unwound because the authorities in it were phantoms.

Then came the Bombay High Court in October 2025. In a matter concerning a faceless tax assessment, the Court quashed an assessment order of about Rs 27.91 crore after finding that it rested on three non-existent precedents, again bearing the signature of generative AI. The pattern was identical: real-looking citations, real financial stakes, zero underlying judgments.

The judicial warnings, though, started earlier. In Christian Louboutin SAS v. M/s The Shoe Boutique-Shutiq (22 August 2023), the Delhi High Court, dealing with a trademark dispute in which a party had leaned on ChatGPT, held plainly that “the responses generated by ChatGPT cannot be the basis of adjudication of legal or factual issues in a court of law.” The Court cautioned against “fictional case laws” and “imaginative data,” and placed AI firmly in the category of a tool for preliminary understanding, not a source of authority. That line has been quoted in courtrooms ever since.

Around the same period, the Punjab and Haryana High Court, in Jaswinder Singh v. State of Punjab (2023), became the first Indian court to openly reference ChatGPT, using it to survey the jurisprudence on bail in cases involving cruelty. The Court was careful to note it was not deciding the matter on the AI’s output; bail was denied on the merits. A year later, in Md. Zakir Hussain v. State of Manipur (order dated 23 May 2024), the Manipur High Court recorded that a government pleader had used ChatGPT to research a service-law point, prompting the Court to flag the reliability question yet again.

And the case that started the global reckoning belongs on this list too, because Indian courts keep citing it. In Mata v. Avianca, Inc. (S.D.N.Y., June 2023), a New York federal court sanctioned two lawyers who had filed a brief stuffed with fake ChatGPT-generated cases, imposing a USD 5,000 penalty under Rule 11 and requiring them to notify the judges falsely named as authors of the invented opinions. It was the world’s first widely reported fake-citation sanction, and it set the template for how courts everywhere would treat the problem: not as an innocent slip, but as a failure of a lawyer’s basic duty to check.

Put these together and the through-line is unmistakable. Across two years, four jurisdictions within India, and stakes ranging from a bail plea to a Rs 669-crore trust, the same mistake keeps producing the same result. Orders get recalled. Assessments get quashed. Lawyers get named. And the thing that would have prevented every single one was a lawyer opening the report before citing it.

The recent Supreme Court guidelines on AI-generated citations

This is the section the whole issue now turns on. Until 2 July 2026, India’s guidance on fake AI citations lived in scattered High Court observations and tribunal recalls. Then the Supreme Court spoke, and it spoke in unusually strong terms.

The case was Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. (2026 INSC 668), decided by Justices P.S. Narasimha and Alok Aradhe on 2 July 2026. It arose from an insolvency application under Section 7 of the Insolvency and Bankruptcy Code, 2016 concerning Essel Infraprojects. The National Company Law Tribunal, and then the appellate tribunal, had decided the matter relying on precedents that turned out to be fabricated: some cases did not exist, and genuine citations had been padded with AI-generated paragraphs that were never part of the real judgments. The Supreme Court set the orders aside and remitted the matter, but it is what the Court said around that outcome that matters for every practitioner.

The Court laid down, in effect, a set of principles that now function as guidelines for the entire legal system. Taken together, they are the strictest statement any Indian court has made on the subject.

The first principle is zero tolerance. The Court held that “it is necessary for Courts to adopt a zero-tolerance mode for producing, citing or using AI-generated precedents without verification.” There is no room in that formulation for a good-faith excuse. Verification is not a best practice the Court hopes advocates will follow; it is a precondition to putting anything before a judge.

The second principle goes directly to the advocate. In the clearest warning yet to the bar, the Court held that “it is a misconduct on the part of an advocate to cite such judgments without verification.” Read that again. The word is misconduct, the same word that anchors disciplinary proceedings under the Advocates Act, 1961. The Court did not merely disapprove of the practice; it labelled it in the vocabulary of professional discipline.

The third principle addresses the judge’s side of the bench. “It is a serious lapse if a judge relies on such a fake or hallucinated AI-generated material as precedents,” the Court said, making clear that the duty to verify runs in both directions, to counsel who cite and to judges who rely.

The fourth principle is about consequences, and it is severe. A decision resting on hallucinated material, the Court held, is “no decision in the eyes of the law, irrespective of whether such material had a direct or indirect bearing.” Crucially, the contamination threshold is minimal: “such decisions are to be set aside even if an iota of fake or hallucinated material enters the decision-making process.” Not a majority of the reasoning. Not the load-bearing citation. An iota. One fabricated authority is enough to unravel the whole order.

The Court also framed the deeper stake. AI, it accepted, can be adopted as an aid to the justice system, but human control over adjudication must remain, in the Court’s words, total and absolute at every stage. The technology assists; it never decides. Fake material entering that process was likened to a contaminant released into the province of law and justice, invisible until it has already done harm.

And then the Court turned to the regulator. It directed the Bar Council of India to constitute a committee to deliberate on the problem of members of the bar placing fake and hallucinated material before courts as if they were genuine precedents. That direction matters for a practical reason: it confirms that, as of mid-2026, the BCI has no AI-specific rules of its own. The Supreme Court has now asked it to build them. Until it does, an advocate’s existing duties of competence and candour, read together with this ruling, are the governing standard.

So what does Pooja Ramesh Singh actually change for a working lawyer? It removes the last shred of ambiguity. Before this ruling, a lawyer caught citing a fake case might have argued it was an honest mistake, an isolated lapse, something short of misconduct. After it, the Supreme Court has already characterised the conduct. Cite an AI-generated authority you did not verify, and you are, on the Court’s own words, in misconduct territory, with your client’s order exposed to being set aside over even a single fabricated line.

High Court AI policies and the draft rules that come next

The Supreme Court’s July 2026 ruling did not land in a vacuum. Over the preceding year, several High Courts had already moved to control AI use inside their own registries, and the Supreme Court’s AI Committee had put out a draft framework signalling where the rules are heading. Reading these together tells you the direction of travel, and it is one way.

Start with the High Court policies, and note carefully who they bind. In July 2025, the Kerala High Court issued what is widely regarded as the first formal AI policy for the district judiciary in India, barring judicial officers from using AI tools to arrive at findings or draft orders, and specifically warning against feeding case data into public cloud-based AI tools, naming the likes of ChatGPT. The Gujarat High Court followed in April 2026 with a comprehensive policy restricting AI in judicial decision-making, and around the same time the Punjab and Haryana High Court circulated guidance to its judicial officers cautioning against tools such as ChatGPT, Gemini and Copilot for judgment-writing and legal research.

Here’s the nuance most commentary misses. Those policies bind judges and court staff, not practising advocates. A lawyer is not disciplined by the Kerala High Court’s internal policy. But treating them as irrelevant to your practice would be a mistake, because they tell you precisely how seriously the judiciary now regards AI-generated content, and how much scrutiny an unverified citation will attract when it reaches the bench.

Now the piece that will affect lawyers most directly: the draft rules. On 3 June 2026, the Supreme Court’s AI Committee released the Draft Regulations for the Use of Artificial Intelligence in Courts, 2026 for public consultation. These are not yet law, and they should be described as a draft, not a binding regime. But their content is a clear map of the future.

The draft permits AI for a defined set of assistive tasks, including legal research, citation verification, drafting assistance, translation, transcription and case management. It insists, in language that echoes the Pooja Ramesh Singh ruling, that every AI system function “solely in an assistive capacity” and never supplant the independent exercise of judicial authority. And, most significantly for practitioners, Regulation 43(3) of the draft provides that where a party or counsel uses AI tools in preparing or submitting any document, pleading or evidence, the AI-assisted character of that document must be disclosed to the court at the time of submission. The draft also contemplates annual audits of court AI systems, AI registers, incident databases, cybersecurity safeguards and data-protection compliance.

If those disclosure provisions survive into the final regulations, the practical calculus changes again. Using AI to help draft a pleading will not be prohibited, but hiding that you did could itself become a problem. The safe posture, even now while the rules are still in draft, is to assume you may one day have to stand behind, and disclose, every AI-assisted document you file, which is one more reason to make sure everything in it is real.

What “sanctioned” actually means for a lawyer

“Sanctioned” is a word that gets thrown around loosely, so it is worth being precise about the concrete consequences an Indian lawyer actually faces when a fake citation surfaces. They stack, and they range from professionally awkward to career-threatening.

The most immediate consequence is that the order collapses. As Pooja Ramesh Singh makes explicit, a decision touched by even an iota of hallucinated material is liable to be set aside. For your client, that means the matter reopens, timelines reset, and the win you thought you had evaporates. The Buckeye Trust recall and the Bombay High Court quashing are that consequence in action. Whatever advantage the fabricated citation seemed to buy is not just lost; it becomes a liability.

The second consequence is cost. Indian courts have increasingly shown a willingness to impose exemplary costs on parties who waste judicial time with fabricated or frivolous material, and AI-generated fake citations sit squarely in that zone. A costs order is the court’s way of pricing the harm of a false filing, and it lands on the party, and often, in practical terms, on the lawyer’s reputation with that client.

The third, and now the sharpest, is professional discipline. The Supreme Court’s characterisation of unverified AI citation as “misconduct on the part of an advocate” is not rhetorical flourish. Professional misconduct is the statutory trigger for disciplinary action under the Advocates Act, 1961, administered through the disciplinary committees of the State Bar Councils and the Bar Council of India. Sanctions on that track can run from reprimand to suspension from practice, and in serious cases to removal from the roll of advocates. The Supreme Court has now told the BCI to build a framework around exactly this conduct. A lawyer who ignores the warning is, quite literally, betting their licence.

Layered on top are the softer but very real costs: the reputational damage of being the advocate whose fake citation is recorded in a published order, the erosion of the court’s trust in your future submissions, and the risk that a client who suffers because of it comes back with a professional-negligence grievance. There is also a latent contempt dimension: knowingly misleading a court is not a trivial matter, and a lawyer who cannot show they verified may struggle to prove the fabrication was innocent.

The uncomfortable truth running through all of this is that the AI takes none of the blame. A language model has no legal standing, no duty to the court, and no licence to lose. Every consequence falls on the human who signed the filing. The tool cannot be sanctioned. You can.

Why verification is now a professional duty, not a courtesy

Some lawyers still treat citation-checking as a nice-to-have, a diligence step you get to if time allows. After Pooja Ramesh Singh, that framing is untenable. Verification is now best understood as a component of the advocate’s core professional duties, and it helps to see exactly which duties it flows from.

The duty of competence comes first. Under the Advocates Act, 1961 and the professional standards in the Bar Council of India Rules, an advocate is expected to bring reasonable skill and diligence to the client’s matter. Placing an authority before a court that you have not confirmed exists is, almost by definition, a failure of diligence. The competence duty does not merely permit verification; it requires it. You cannot competently rely on a case you have not read.

Close behind is the duty of candour to the court. An advocate must not mislead the court, whether deliberately or through recklessness. A fabricated citation misleads the court about the state of the law, and doing so because you outsourced your research to a tool you did not check is no defence. The court is entitled to assume that every authority in your submission is one you stand behind.

There is a confidentiality dimension too, which pulls in the same direction. Pasting privileged client facts into a consumer AI tool can, on free and consumer tiers where prompts may be retained to train the model, sit uneasily with an advocate’s duty of confidentiality and with the professional-communication protection recognised in Section 132 of the Bharatiya Sakshya Adhiniyam, 2023, the successor to the old evidence-law privilege. Both the Kerala and Gujarat High Court policies warn specifically against feeding case data into public cloud AI. As the core obligations of the Digital Personal Data Protection Act, 2023 come fully into force, expected around 2027, the discipline of keeping client data out of unvetted tools moves further from etiquette toward legal duty.

The unifying principle is accountability. AI has no legal recognition in the courtroom; the advocate alone is answerable for the work product. Think of the AI as a brilliant but unreliable junior: fast, tireless, occasionally the author of a confident fiction, and never to be left unsupervised. You would not file a junior’s research memo without reading it. The model deserves less trust, not more.

How to avoid fake citations: a verification workflow that works

Enough about the danger. The reassuring part is that avoiding hallucinated case law is not hard. It does not require abandoning AI, and it does not require a subscription to anything you do not already have. It requires a workflow, applied every time, with no exceptions on busy days. Here is one that holds up.

Step 1: Never ask the AI for the law from memory. This is the golden rule, and it prevents most hallucinations at the source. There are two ways to use an LLM in legal work, and they have opposite risk profiles. “Reason over what I paste,” where you supply the judgment, statute or contract and ask the model to summarise, compare or draft from that text, is safe. “Tell me the law,” where you ask the model to recall authorities from its own memory, is where fabrication lives. Stay on the first side of that line and you have already closed off the main failure mode.

Step 2: Source your authorities yourself, from a real database. Run your actual searches on SCC Online, Manupatra or Indian Kanoon and pull the genuine judgments. This is the step no general-purpose AI can do for you, and the step that keeps you honest. If you want the AI’s help framing search terms or Boolean queries, fine, but the retrieval of real cases happens in an authorised corpus, by you.

Step 3: Feed the real material back for analysis. Now the AI does what it is genuinely excellent at. Paste the genuine judgments in and ask it to extract the ratio, compare holdings, build a table, or identify which case best supports your point, instructing it to use only the text you supplied and to add nothing that is not in that material. Used this way, the tool compresses hours of reading into minutes, with no room to invent.

Step 4: Verify every citation independently, before it goes anywhere. Even when you think you sourced a case properly, confirm each element against the authorised report: the party names, the neutral citation, the volume and page, and, critically, that the paragraph you are quoting actually says what you claim. Treat an unverified citation as if it does not exist. The rule reduces to five words: never cite what you have not opened.

Step 5: Keep client data out of consumer tools, and keep a human on every conclusion. Anonymise facts and strip identifiers before pasting anything sensitive, and prefer enterprise tiers with data-retention controls for confidential work. And never let an AI-generated legal conclusion go out under your name without a human, you, having checked and owned it.

Run every AI-assisted work product through a short pre-filing checklist and the risk drops close to zero. Has every citation been confirmed in an authorised database? Has every quoted paragraph been read in the original? Is the position current, given the model’s knowledge cutoff? Was any privileged client data kept off consumer tiers? And are you prepared, if a court asks, to stand behind and disclose the AI assistance you used? Five questions. A few minutes. A career protected.

Common mistakes that put fake citations in front of judges

Even lawyers who know the risk fall into predictable traps. Recognising them is half the defence.

The most common is trusting the format. A citation that looks perfectly formed, right party-name shape, right citation pattern, confident paragraph number, feels verified when it is not. Format is not verification. The only thing that verifies a case is opening the report and reading it. Fabricated citations are convincing precisely because the model is excellent at mimicking the shape of a real one.

A close second is the “it’s probably fine” shortcut under deadline pressure. The fake-citation disasters almost always trace back to a moment when a lawyer, short on time, decided that checking each authority was a luxury. That is exactly when the discipline matters most. The busy filing is the dangerous filing.

Third is asking the AI to verify its own citations. A tempting move, and a useless one. A model that invented a case will often “confirm” it just as confidently, or invent a fresh detail to support the first fabrication. Verification has to happen in an independent, authorised source, never inside the same tool that produced the citation.

Fourth is over-trusting India-specific AI tools. Grounded, retrieval-based products like Manupatra’s and SCC Online’s assistants are meaningfully safer than a raw chatbot, because they search a real corpus. But “lower risk” is not “no risk,” and the professional duty to verify does not switch off because the tool is a legal one. Read the case the tool surfaced before you cite it.

The fifth mistake is treating the knowledge cutoff as if it did not exist. Ask a model about a 2026 amendment or a recent judgment and it may confidently describe the pre-amendment position, or invent a development, because it simply does not know what happened after its training cutoff. For anything time-sensitive, currency has to be checked against the live law.

The common thread? Every one of these is a decision to trust the machine over the record. Reverse that instinct, treat the authorised report as the only truth and the AI as a helpful but unreliable assistant, and the traps lose their power.

Conclusion

The most useful way to hold all of this in your head is simple. AI is a genuine force multiplier for legal research and drafting, and the lawyers who use it well are quietly out-working those who do not. But it is a tool that occasionally invents authority with total confidence, and Indian courts have run out of patience with the results. The Supreme Court’s 2 July 2026 ruling in Pooja Ramesh Singh drew the line in the hardest language it could: zero tolerance, misconduct, no decision in the eyes of the law, set aside on even an iota of fabrication.

Frequently asked questions

What is a hallucinated citation in law?

A hallucinated citation is a case reference, judgment name or quoted paragraph that an AI tool produces confidently but that does not exist or is false. It may be an entirely invented case, a real case with fabricated content, or a genuine principle wrongly attributed to the wrong judgment. The term reflects the tendency of large language models to generate fluent, plausible text rather than retrieve verified records.

Can lawyers in India be punished for citing fake AI-generated cases?

Yes. In Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. (2026 INSC 668), decided on 2 July 2026, the Supreme Court held that citing AI-generated judgments without verification is misconduct on the part of an advocate. Professional misconduct is the statutory basis for disciplinary action under the Advocates Act, 1961, which can range from reprimand to suspension. Separately, orders built on such citations can be set aside and cost orders imposed.

What did the Supreme Court say about AI-generated citations in 2026?

The Court adopted a zero-tolerance approach to producing, citing or using AI-generated precedents without verification. It held that a decision resting on hallucinated material is no decision in the eyes of the law, and that such a decision must be set aside even if only an iota of fabricated material entered the reasoning. It also directed the Bar Council of India to constitute a committee to address advocates placing fake material before courts.

Are lawyers allowed to use ChatGPT for legal research in India?

There is no law or Bar Council of India rule prohibiting an advocate from using ChatGPT or similar tools for research or drafting. What the law requires is that the advocate independently verify everything the tool produces and take full responsibility for it. The High Court AI policies that restrict such use apply to judges and court staff, not to practising advocates.

How can I verify whether an AI-generated case citation is real?

Confirm every element of the citation in an authorised legal database such as SCC Online, Manupatra, or the official law reports, or on Indian Kanoon. Check that the case exists, that the party names and neutral citation match, and that the paragraph you intend to quote actually appears in the judgment and says what you claim. Never rely on the AI tool to verify its own citation, and never cite a case you have not opened and read.

Do the Supreme Court’s draft AI rules require lawyers to disclose AI use?

The Draft Regulations for the Use of Artificial Intelligence in Courts, 2026, released for public consultation on 3 June 2026, propose that the AI-assisted character of any document, pleading or evidence be disclosed to the court at the time of submission. These regulations are still in draft form and not yet binding, but they indicate the direction of future practice, so treating AI-assisted filings as potentially disclosable is the prudent approach.

Which is safer for legal research, ChatGPT or a dedicated Indian legal AI tool?

Dedicated Indian legal AI tools such as Manupatra’s and SCC Online’s assistants are generally safer because they use retrieval-augmented generation, answering from a real corpus of Indian judgments rather than from memory, which reduces hallucination. General-purpose tools like ChatGPT and Claude have no such corpus unless you supply the source text. Either way, the professional duty to verify every citation in an authorised report remains the same.

References