AI in Indian Courts: Can Judges and Lawyers Trust Artificial Intelligence?
- Manoj Ambat
- Aug 13
- 14 min read
Artificial intelligence has already entered the legal profession. Lawyers use it to search authorities, summarise judgments, analyse documents, prepare drafts, compare statutes and explore arguments. Courts themselves are experimenting with digital tools, transcription systems, research assistance and other forms of technological support. The question, therefore, is no longer whether artificial intelligence will enter the justice system. It already has. The more difficult question is where the boundary should be drawn between using AI as an assistant and allowing AI to influence the substance of judicial decision-making.
That question has now moved from theoretical debate into the courtroom in India. In July 2026, the Supreme Court of India delivered a significant judgment in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668, after discovering that an adjudicatory decision had relied upon non-existent, fake and AI-hallucinated legal material presented as precedent. The Supreme Court set aside the orders of the NCLT and NCLAT and made a broader point that could become one of the defining principles of Indian judicial AI jurisprudence: technology may assist adjudication, but human beings must retain control over adjudication at every stage.
This is an important moment because the problem was not simply that a computer made a mistake. Computers have always made mistakes. Databases contain errors, search engines return irrelevant material and software occasionally produces defective results. What makes generative artificial intelligence fundamentally different is its ability to produce information that looks authoritative even when it is false. An AI system can generate a case name that sounds perfectly plausible, attach a convincing citation to it, describe a judgment that never existed and even create paragraphs that appear to come from that judgment. To a reader who does not independently verify the material, the fabrication may look indistinguishable from genuine legal research.
And that creates a uniquely dangerous problem for law.
The Problem Is Not That AI Can Be Wrong
Law operates on authority. A lawyer does not merely present an attractive argument; the lawyer must ordinarily connect that argument to statutes, precedents, rules, constitutional principles and established legal reasoning. A judge, in turn, must provide reasons capable of scrutiny by the parties, appellate courts and the public. The legitimacy of the judicial process therefore depends not merely upon reaching a conclusion but upon reaching it through a process that can be examined and justified.
An AI hallucination attacks that foundation at its source.
If an artificial intelligence system invents a restaurant, the consequence may be inconvenience. If it invents a historical event, the consequence may be misinformation. But if it invents a Supreme Court judgment and a court relies upon that fictional authority, the problem becomes institutional. The fabricated authority enters the reasoning process of the State itself.
The Supreme Court's 2026 decision is therefore significant because it treats AI-generated fake legal material as a threat to the integrity of adjudication, rather than as an ordinary research error. The Court's landmark judgment summary records that the case concerned six AI-generated citations that were either wholly non-existent or attributed with paragraphs that did not exist.
This distinction matters enormously.
A wrong legal interpretation can be challenged on appeal. A genuine precedent can be distinguished, overruled or reconsidered. A poorly researched argument can be rejected by a court. But a fictional precedent creates something fundamentally different: there is no underlying legal authority to examine at all.
The legal system is then reasoning from a nonexistent foundation.
The “Human in the Loop” Principle
The most important part of the Supreme Court's judgment may ultimately prove to be broader than the facts of the particular insolvency dispute.
The Court did not reject artificial intelligence as such. Instead, it recognised the potential of AI technology to assist adjudication while insisting upon human control at every stage.
That approach is considerably more sophisticated than either extreme of the present AI debate.
One extreme says that AI should be allowed to transform legal decision-making because machines can process enormous quantities of information faster than humans. The opposite extreme says that artificial intelligence has no legitimate place in judicial work because justice is inherently human. Neither proposition adequately describes the emerging reality.
The better model is AI-assisted justice, not AI-determined justice.
Under this model, artificial intelligence can perform tasks at which machines are exceptionally efficient. It can search enormous collections of documents. It can identify patterns. It can compare thousands of pages. It can organise authorities. It can help identify inconsistencies in pleadings. It can transcribe proceedings and assist with administrative tasks. The Supreme Court itself has been pursuing technological transformation, including AI-related judicial technology, while its 2025 White Paper on Artificial Intelligence and Judiciary demonstrates that the issue has been under institutional consideration for some time.
But the machine cannot be allowed to silently cross the line from information processing to judicial authority.
A judge may ask an AI system to help locate relevant authorities. The judge cannot treat the machine's answer as authority without verification. A lawyer may use AI to generate a first draft. The lawyer cannot file the output without independently checking the law. A court may employ AI-assisted tools. But the judicial reasoning must remain attributable to the judge.
The distinction is simple but fundamental:
AI can assist the reasoning process. It cannot become the source of legal legitimacy.
Why Lawyers Face a Special Responsibility
The emergence of AI changes the professional responsibility of advocates.
For decades, legal research required lawyers to consult law reports, databases, statutes, commentaries and judgments. The basic professional obligation was therefore relatively intuitive: verify the authority before relying upon it.
Generative AI changes the workflow because it creates the temptation to reverse that sequence.
A lawyer can now ask a machine a question and receive an apparently complete answer in seconds. The answer may contain case names, citations, quotations and legal propositions. The presentation itself can create an illusion of reliability.
That is precisely where professional discipline becomes essential.
An advocate cannot outsource the duty of verification to an algorithm.
The machine may have produced the research, but the lawyer puts the proposition before the court. The lawyer signs the pleading. The lawyer makes the submission. The lawyer owes the professional duty to the court.
This is why the Supreme Court's approach is likely to have consequences extending beyond judges and tribunals. It reinforces a broader principle: technological assistance does not transfer professional responsibility from the human professional to the technology.
If anything, AI increases the responsibility of the lawyer because the lawyer now has access to a tool capable of producing enormous quantities of plausible but unreliable material.
The faster the tool becomes, the more disciplined the verification process must become.
The New Legal Research Rule: Trust Nothing Until Verified
The traditional legal research model can be described as:
Search → Read → Analyse → Cite.
The AI-assisted model needs another step:
Prompt → Generate → Verify → Read → Analyse → Cite.
That additional verification stage cannot be treated as optional.
The lawyer must verify that the case actually exists, that the citation is correct, that the judgment says what the AI claims it says, that the quoted paragraph exists, that the judgment has not been overruled and that the proposition remains legally applicable.
This may appear obvious to experienced advocates. But the problem is likely to become more serious as AI becomes increasingly integrated into legal research platforms.
The danger will not necessarily come from obviously absurd AI output. The most dangerous hallucinations will be plausible ones.
A fictional case carrying a bizarre name may be detected immediately. A fictional judgment with a realistic title, correct-looking court designation and apparently appropriate citation is much more difficult to identify.
That creates a new professional discipline: AI-assisted legal source verification.
In the future, competent legal practice may require not merely knowing how to use AI, but knowing how to audit AI.
What Happens When AI Enters Judicial Reasoning?
This brings us to the more difficult constitutional and institutional question.
Suppose an AI system analyses thousands of previous judgments and identifies a pattern. It suggests that a particular interpretation is consistent with ninety percent of previous decisions. The judge considers that analysis and ultimately adopts the conclusion.
Has AI merely assisted research?
Perhaps.
But now imagine that the AI system ranks competing interpretations, predicts the likely outcome, identifies which authorities should be treated as persuasive and recommends a particular conclusion. The judge accepts the recommendation.
At what point has assistance become influence?
And if the AI system is trained on material whose selection and weighting are unknown, who is responsible for the resulting reasoning?
These questions go directly to judicial accountability.
A judge can explain why a particular precedent was followed. A judge can distinguish facts. A judge can explain constitutional principles. A judge can defend the reasoning before an appellate court.
But can a court meaningfully defend a conclusion that was substantially shaped by an opaque algorithm whose internal reasoning cannot be fully examined?
This is why the concept of the human in the loop is so important. It is not simply a technological safeguard. It is a legal and constitutional safeguard.
Human control preserves accountability.
AI and the Problem of Evidence
The problem becomes even more complicated when AI moves from legal research into evidence.
A court historically asks whether a document, photograph, recording or electronic communication is genuine. Digital technology has already made authentication more complicated. Generative AI takes the problem to another level because the existence of a digital file no longer necessarily tells us anything reliable about the event it appears to represent.
A video can be synthetically generated. A voice can be cloned. An image can be manipulated. A document can be fabricated. A person's face can be inserted into material in which they never appeared.
The question is no longer merely:
“Is this digital file authentic?”
It becomes:
“How do we establish what part of this digital file represents reality?”
India has already moved toward stronger regulation of synthetic and AI-generated content. The 2026 amendments to the Information Technology Rules have introduced new obligations concerning synthetic information and platform responses, reflecting the increasing regulatory recognition of AI-generated deception.
Indian courts are also confronting deepfake disputes in practical litigation. Recent proceedings demonstrate that courts are increasingly being asked to provide urgent remedies against AI-generated impersonation, synthetic videos and misuse of identity.
The evidentiary implications are enormous.
Tomorrow's lawyer may therefore need to prove not only that an electronic record exists, but also that the record has not been synthetically manipulated.
The courtroom of the AI era may require a new layer of digital provenance.
The Coming Battle Over Authenticity
This could become one of the defining legal questions of the next decade.
For centuries, law developed techniques for determining authenticity. Signatures, witnesses, seals, handwriting, official records and documentary chains were all mechanisms through which courts established trust.
Digital technology introduced cryptographic signatures, metadata, electronic records and forensic examination.
Generative AI now creates an uncomfortable possibility: a perfectly convincing piece of evidence may be completely false.
That means authenticity can no longer depend primarily upon appearance.
A video may look real and still be fake. A voice may sound identical and still be synthetic. A document may contain the correct logos and formatting and still never have existed.
The legal system will therefore have to move from a culture of appearance-based authenticity toward process-based authenticity.
Where did the file originate?
Who created it?
How was it stored?
Has it been altered?
What metadata exists?
Can its provenance be independently established?
What technological methods were used to verify it?
Can the opposing party meaningfully challenge the verification process?
These questions will increasingly become part of ordinary litigation.
India's Emerging Regulatory Response
India is not approaching this challenge from a blank slate.
The Supreme Court has already been examining AI and the judiciary through institutional research and policy initiatives. In June 2026, the Court invited comments on draft regulations concerning the use of artificial intelligence in courts, indicating that the question is moving from experimentation toward formal governance.
This is an important development because judicial AI cannot be governed solely through informal technological enthusiasm.
The justice system requires rules concerning confidentiality, verification, disclosure, accountability, cybersecurity, data protection, bias, transparency and human oversight.
Consider confidential litigation material. A lawyer may upload pleadings, medical records, financial statements, privileged communications or commercially sensitive documents into an AI system. If that system operates through an external platform, questions immediately arise concerning where the data goes, how it is processed, whether it is retained and whether it can be used for further model development.
Technology therefore creates not one legal issue but an interconnected chain of issues.
AI use in law touches professional ethics, evidence, privacy, confidentiality, cybersecurity, intellectual property, procedural fairness and judicial independence simultaneously.
That is why AI law cannot remain a narrow technology-law specialty.
It is becoming a foundational component of modern legal practice.
Should AI Use Be Disclosed in Court?
One of the most interesting emerging questions is whether lawyers should be required to disclose when artificial intelligence has materially contributed to a pleading or submission.
The issue is not necessarily whether a lawyer used a spell-checker or a basic research tool. The question becomes more significant when generative AI has drafted substantive arguments, identified authorities or produced factual propositions.
Disclosure could serve several purposes.
First, it could encourage professional accountability. A lawyer who knows that AI use must be disclosed is more likely to verify the output.
Second, it could allow the court to assess the reliability of the material.
Third, it could help develop professional standards by allowing courts and regulators to understand how AI is actually being used.
But disclosure alone is not enough.
A statement saying “AI was used in preparing this submission” does not solve the hallucination problem.
The real safeguard is verification.
A lawyer should ultimately be able to stand behind every legal proposition placed before the court, regardless of whether the first draft came from a junior, a database or an artificial intelligence system.
Could AI Ever Become a Judge?
This is where the debate becomes philosophical.
If artificial intelligence eventually becomes capable of analysing every relevant precedent, statute, fact and argument faster than a human judge, why should it not decide cases?
The answer cannot simply be that humans are more intelligent.
Machines may eventually outperform humans in many forms of analytical processing.
The deeper reason is that judging is not merely an exercise in information processing.
Judicial decision-making involves interpretation, proportionality, institutional legitimacy, constitutional values, empathy, contextual judgment and responsibility. A court does not simply calculate an answer. It exercises public power.
A judgment carries authority because a constitutionally authorised human institution has taken responsibility for the decision.
That distinction matters.
A machine may be capable of telling us what the law appears to require according to historical patterns. But deciding what the law means in a novel constitutional situation may require a form of normative judgment that cannot simply be reduced to statistical prediction.
And even if a machine could make such judgments, society would still have to answer the question:
Who is accountable for the decision?
That is ultimately the strongest argument for keeping the human at the centre.
The Future Is Not Human Versus AI
The most useful way to understand the future of legal technology is not as a contest between humans and machines.
It is a question of division of responsibility.
Machines are extraordinarily good at scale.
Humans remain responsible for legitimacy.
AI can search millions of pages.
A lawyer decides which authorities matter.
AI can identify patterns.
A judge determines which pattern is legally relevant.
AI can generate arguments.
An advocate decides which arguments can ethically be presented.
AI can analyse evidence.
A court determines whether the evidence is legally admissible and reliable.
AI can assist reasoning.
The human decision-maker remains accountable for the judgment.
This division is likely to become the most sustainable model for AI-assisted justice.
The Real Danger Is Automation Without Accountability
The greatest danger is therefore not artificial intelligence itself.
It is uncritical dependence on artificial intelligence.
A lawyer who treats an AI answer as legal authority is effectively replacing legal research with algorithmic trust.
A judge who accepts an AI-generated conclusion without independent verification risks delegating judicial reasoning.
A litigant who produces synthetic evidence risks corrupting the truth-finding function of the court.
A technology provider that cannot explain how its system handles sensitive legal data creates a different category of institutional risk.
The central principle should therefore be simple:
The more powerful the AI system becomes, the stronger the human accountability surrounding its use must become.
This is particularly important in India because the country's justice system is dealing simultaneously with enormous caseloads, technological transformation, expanding digital evidence and rapid adoption of generative AI. AI offers genuine opportunities to reduce administrative burdens and improve access to information. But efficiency cannot become a justification for weakening the safeguards that make judicial decisions legitimate.
Justice cannot simply be made faster.
It has to remain trustworthy.
The ALI Perspective: Technology Can Assist Justice, But It Cannot Own Justice
The Supreme Court's 2026 AI-hallucination judgment may eventually be remembered as one of the early warning signals of India's AI-judiciary era.
Its significance lies not merely in the fact that fake case law entered a judicial decision. The deeper significance is that the incident forces the legal system to define the proper relationship between human judgment and machine intelligence.
India does not need to reject AI.
It should embrace it carefully.
Artificial intelligence can help lawyers work faster, help courts manage information and potentially improve access to justice. It can become an extraordinary legal research assistant and an important administrative tool.
But the legal profession must resist one dangerous temptation: confusing plausibility with truth.
An AI system can produce a convincing answer without possessing legal authority.
It can produce a persuasive argument without understanding its consequences.
It can generate a citation without knowing whether the case exists.
And it can produce an answer with extraordinary confidence while being completely wrong.
That is why the future of legal AI should not be built around the question, “How much of the lawyer can AI replace?”
The better question is:
“How much better can lawyers and courts become when AI is placed under responsible human control?”
That distinction could determine whether artificial intelligence strengthens the justice system or quietly undermines the foundations upon which it rests.
The courtroom of the future will almost certainly contain artificial intelligence.
The essential question is who will remain in charge.
At ALI, our answer is clear: AI may assist justice. It must never become the authority that defines justice.
Conclusion
The arrival of AI in Indian courts is no longer a distant technological possibility. It is already producing real cases, real regulatory questions and real consequences. The Supreme Court's decision in Pooja Ramesh Singh demonstrates that the legal system will not tolerate the substitution of fictional machine-generated authority for genuine legal precedent. At the same time, the Court's willingness to explore responsible AI assistance shows that India is not choosing between technology and tradition. It is attempting to build a controlled intersection between the two.
The challenge ahead will be to develop rules that preserve the strengths of artificial intelligence without surrendering the values that make adjudication legitimate: verification, accountability, transparency, independence and human judgment.
The future lawyer will almost certainly use AI.
The future judge may also use AI.
But the future of justice should never depend upon whether an algorithm sounds convincing.
It must depend upon whether the law can still be verified, reasoned, challenged and ultimately owned by a responsible human decision-maker.
CITATIONS
1. Supreme Court's official landmark judgment summary — essential
The Supreme Court itself lists Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668, decided 2 July 2026, specifically under the subject of reliance on AI-generated fake and hallucinated case law. It confirms that six citations relied upon by the NCLT/NCLAT were either wholly non-existent or contained non-existent paragraphs.
2. The actual Supreme Court judgment — our primary authority
This is even more important. The judgment states that the Court set aside the NCLT and appellate judgment to preserve the integrity of adjudication, while declaring its intention to adopt AI in aid of adjudication with “a human in the loop at every stage.”
3. The judgment's detailed findings on the six citations
This is excellent for the section discussing hallucinations. The Court independently examined the authorities and found examples including a non-existent citation, a wrong citation, and genuine cases attributed with non-existent paragraphs.
4. The Supreme Court's zero-tolerance position
The judgment expressly says courts should adopt a zero-tolerance approach to producing, citing or using AI-generated precedents without verification. It also treats an advocate's citation of such material without verification as misconduct and reliance by a judge as a serious lapse.
That is a particularly powerful citation for our “What does this mean for lawyers?” section.
5. Supreme Court's AI-in-courts regulatory initiative
This is important because our article discusses the future regulatory framework. The Supreme Court's official notices show that it invited comments and subsequently extended the deadline for comments on the draft “Regulations for Use of Artificial Intelligence (AI) in Courts, 2026.”
6. Department of Justice — AI and the judiciary
For the section about India's broader digital-justice transition, we can cite the Ministry of Law and Justice's 2026 parliamentary response concerning AI use in the judiciary, including case management, legal research, translation, transcription and safeguards concerning errors, bias, judicial independence and accountability.
Comments