Supreme Court Warns Against Unregulated Use of AI in Judicial Decisions

Why in news ?

The Supreme Court set aside a judgment of the National Company Law Tribunal (NCLT) after finding that it relied on fake, non-existent, and AI-generated hallucinated judgments/precedents.

Key Observations of the Supreme Court

• AI can assist judicial processes, but adjudication must remain under the complete control of human judges.
• Reliance on AI-generated fake precedents undermines the credibility of the justice system.
• The apex bar body has been directed to frame guidelines and disciplinary measures to regulate AI use in legal practice.

Draft Regulations on AI Use in the Judiciary

• AI may be used only for administrative functions, such as :
  • Case management
  • Cause list preparation
  • Hearing scheduling
  • Court transcription
  • Translation of judgments
• AI cannot be used for :
  • Risk scoring
  • Bail eligibility assessment
  • Predicting recidivism
  • Assessing witness credibility
• Processing of personal data must comply with the Digital Personal Data Protection Act, 2023.
• AI systems must not perpetuate bias based on race, religion, caste, gender, disability, language, or economic status.
• AI adoption should not widen the digital divide.
• A dedicated Apex Body is proposed at the Supreme Court to oversee AI adoption in the judiciary.

Current Use of AI in India's Judiciary

• Transcription of oral arguments
• Translation of judgments (SUVAAS)
• Detection of defects in e-filing
• Legal research
• Metadata extraction
• AI technologies such as Machine Learning (ML), Optical Character Recognition (OCR), and Natural Language Processing (NLP) are integrated into the e-Courts Project.

Significance of AI in the Judiciary

• Helps reduce the backlog of pending cases.
• Improves prison and investigation management.
• Enhances multilingual access to judicial documents.
• Increases efficiency and accuracy in legal research.
• Improves access to justice through AI-powered legal assistance.

Key Concerns

• AI hallucinations and fabricated legal precedents.
• Algorithmic bias and over-reliance on AI.
• Data privacy and security concerns.
• Inadequate digital infrastructure.
• Lack of human empathy and judicial discretion in sensitive cases.

Way Forward

• AI should function only as a decision-support tool, not a decision-maker.
• Robust regulatory, ethical, and data protection frameworks should be established.
• Human oversight must remain central to ensure transparency, accountability, and fairness in judicial decision-making.