Artificial intelligence (AI) may make legal research faster, improve efficiency and transform the administration of justice, but it cannot replace the human conscience, wisdom and accountability required by South Africa’s constitutional democracy.
This was the central message emerging from the third and final day of the University of Johannesburg (UJ)’s inaugural AI and the Law Conference last week. The conference was held between 9–11 July at Skukuza Safari Lodge in Mpumalanga. Opening the final day’s keynote proceedings, retired Supreme Court of Appeal Justice Boissie Henry Mbha delivered a strong warning against allowing technological efficiency to undermine the constitutional values on which South Africa’s justice system is built.
“We need to manage this new technology carefully so that it supports rather than replaces the human heartbeat of justice in South Africa,” Justice Mbha said. “This new technology must supplement, not supplant, what we have to do.”
His address brought the conversation back to South Africa’s Constitution, arguing that every AI system deployed in the public or private sector must remain subject to the rule of law, particularly when automated decisions affect people’s rights, dignity or legal status.
“Will artificial intelligence serve as an instrument to democratise justice, or will it become a tool that reinforces exclusion and erodes judicial integrity?” he asked.
Justice Mbha warned that AI systems trained on historical data risk reproducing existing inequalities and discrimination. He argued that opaque automated decisions could infringe constitutional rights to equality and human dignity, particularly when people are subjected to life-changing decisions without a clear explanation or meaningful human consideration.
“Being subjected to a life-altering decision made by a black-box algorithm without any human empathy or individualised consideration reduces a human being to a mere statistic,” he said.
He also addressed South Africa’s regulatory landscape, noting the absence of dedicated AI legislation. While existing laws can provide some safeguards, Justice Mbha argued that the rapid development of AI continues to raise complex questions about accountability, transparency and responsibility.
One of his strongest warnings concerned the growing use of generative AI in legal practice. Drawing on cases in South Africa and abroad where lawyers submitted fictitious AI-generated legal authorities to courts, he stressed that technology does not remove a legal practitioner’s professional duty to verify information.
“The charm of AI in legal practice lies in its misleading competence,” he said. “Generative AI models do not possess an understanding of truth. They possess an understanding of statistical probability.”
He was particularly firm on the role of AI in judicial decision-making, arguing that delivering judgment requires far more than processing data or identifying statistical patterns.
“Judicial decision-making is not a mechanical exercise in data processing. It is an act of human conscience, deep ethical reflection and contextual evaluation,” he said.
“Writing a judgment, you’re dealing with people. You can’t delegate that to a machine.”
Justice Mbha argued that while AI could support the justice system, it could never fully understand the lived reality of an impoverished litigant or the African philosophy of Ubuntu that informs South Africa’s constitutional jurisprudence.
“While technology can help our legal system work faster, it can never replace the human wisdom, fairness and sense of Ubuntu that our Constitution requires,” he said.
The conversation on regulation continued with Professor Mzukisi Njotini, Dean of the Faculty of Law at the University of Fort Hare, who examined the challenges posed by opaque AI systems and what he described as the “black box” phenomenon.
Prof Njotini argued that society stands at a crossroads. While AI offers significant opportunities to improve productivity, responsiveness and access to justice, the inability to understand how some systems reach their decisions creates serious challenges for law and accountability.
“For regulation, this opacity is not merely a design or technical error. It is rather a regulatory control or governance calamity,” he said.
The fundamental question, he argued, was straightforward but difficult to answer: who will be held accountable when things go wrong?
According to Prof Njotini, the opacity of AI can disrupt the chain of legal liability, make it difficult to establish intention and fault, conceal systematic discrimination and create an illusion of accuracy. This presents a particular problem for the law, which relies heavily on certainty, predictability and explainability.
“Everything that happens in this black box frustrates the law because there is no certainty. Law operates on certainty and explainability,” he said.
Rather than relying on static regulations that struggle to keep pace with technological advancement, Prof Njotini called for more dynamic approaches to AI governance and stronger collaboration between lawyers, programmers and technology developers.
He argued that trustworthy AI should be lawful, ethical and robust, with human oversight remaining central to its development and deployment.
“We have to be human-centric as regulators. There has to be human oversight. There has to be human vigilance,” he said.
The two keynote addresses reinforced one of the central messages that emerged throughout the three-day conference: technological advancement cannot come at the expense of constitutional rights, human dignity, accountability or the rule of law.
As Justice Mbha concluded by saying, “Protecting judicial independence means drawing a clear line where technology stops and human judgment begins. By keeping the power of decision-making in the hands of independent judges, we ensure that our legal system remains fair, transparent and deeply committed to serving the people of South Africa.”
Beyond the keynote addresses, the final day continued with parallel research sessions, providing academics, researchers and legal practitioners with a platform to present their work and contribute to the growing body of scholarship on AI, law, governance and sustainable development.
The final day also featured an exclusive screening of an episode of the Beyond the Code: AI and Law podcast featuring Prof Mpedi, further extending the conference’s conversations on the evolving relationship between artificial intelligence, law and society.
Bringing the three-day conference to a close, Deputy Vice-Chancellor: Research & Innovation, Professor Refilwe Phaswana-Mafuya, reflected on the depth of engagement and called on delegates to ensure that the conversations initiated at Skukuza translate into lasting academic, policy and practical outcomes.
“Over the past three days, we have engaged in rich, thought-provoking, intensive, mind-boggling, powerful and information-packed sessions,” she said.
Prof Phaswana-Mafuya emphasised that the relationship between AI and the law is no longer an abstract or theoretical concern, but an urgent reality requiring active engagement from lawyers, academics, policymakers, industry and society.
“We have learned a lot about the relationship between AI and the law and that this is not an abstract or theoretical relationship. It is a real relationship. It is urgent. It is practical. It is here,” she said.
“This conference has demonstrated that Africa and South Africa have both the expertise and leadership to shape the global conversation on artificial intelligence.”
She also acknowledged UJ Vice-Chancellor and Principal Professor Letlhokwa George Mpedi for his foresight and leadership in establishing the AI and the Law Institute, and Executive Dean of the Faculty of Law Professor Charles Maimela and the organising team for bringing the conference to life.
Prof Phaswana-Mafuya stressed that the inaugural conference was not an end in itself, but the beginning of sustained research, collaboration and engagement. She confirmed that the conference would return for a second edition and encouraged delegates to develop their research papers for consideration in a forthcoming edited volume, Artificial Intelligence and the Law: Advancing the Sustainable Development Goals through Law, Governance and Practice.
“This publication will ensure that the important ideas and collaborations initiated here continue to shape scholarship, policy and practice well beyond this conference,” she said.
Closing with a commitment to the future, Prof Phaswana-Mafuya said: “We opened this conference with a purpose, and we are closing it with a commitment.”
