Deus Ex Machina?

Generative AI and the duty to verify

By Matt Lewis SC and Alessandro Cowley

Reported cases on the use of generative AI in litigation tend to point one way. The AI chatbot hallucinates, a court exposes the error, and public admonishment follows. If the culprit is a lawyer, there may also be serious professional consequences. Unless the lawyer self-refers to the relevant professional regulator, there is a real risk that a court will refer that lawyer for contempt of court.

According to Damien Charlotin’s database, there are currently no fewer than 1,596 examples of case hallucinations worldwide: seventy four are Australian, seven are from New Zealand, and fifty nine are from the United Kingdom.

The latest example is Cork v Smith[1] – the 59th from the United Kingdom. There, a junior lawyer at the respected international firm Pinsent Masons used an AI chatbot piloted by the firm in an uncontested matter. The chatbot hallucinated the text of a procedural rule but cautioned that it was not confident the text was accurate, and that verification was appropriate. Unfortunately, the junior lawyer failed to heed that caution when drafting correspondence to the court. The senior associate and partner in charge – each of whom was unaware that AI had been used – largely waved the correspondence through. After the court exposed the hallucination, the chatbot offered to draft an apology. The junior lawyer rejected that offer. Disaster ensued. Understandably frustrated, the court ordered the authors of the communications to explain themselves in evidence. In addition to the inevitable public admonishment, Pinsent Masons referred itself, the senior associate and the partner to the Solicitors Regulation Authority (SRA). The junior lawyer – anonymised by the firm – later identified himself as the junior lawyer to the SRA. That, combined with full disclosure of the AI transcript and frank apologies, probably saved the junior lawyer from a referral for contempt. The judgment did not pull any punches and does not make for pleasant reading.

Facts

It began as routine “boxwork” – what Antipodeans might recognise as judicial administrative paperwork. It was an uncontested “block transfer application” to remove one insolvency practitioner and appoint another. Applications of that kind are usually made when the incumbent office-holder dies, retires or is otherwise unwilling to continue. As an aside, such applications are no longer made to the Chancery Division of the High Court, but to the Business and Property Courts, Insolvency and Companies List. In any event, the application was the kind of administrative matter ordinarily disposed of “on the papers”.

Pinsent Masons filed an application on behalf of its client seeking the removal of the administrator/liquidator in a small number of administrations and voluntary liquidations. Notably, the draft order also sought to release the same insolvency practitioner from liability. Whether such an order could be made is not straightforward; it ordinarily requires an application to the Secretary of State. On 20 March 2026, ICC Judge Mullen requested that Pinsent Masons identify the power that permitted the Court to grant the release.

By letter dated 30 March 2026, Pinsent Masons advised the Court that the power was conferred by r 12.37(5) of the Insolvency (England and Wales) Rules 2016. It quoted the provision in italics and, for all intents and purposes, the passage resembled an extract from the Rules. The letter was signed by a senior associate and the partner with carriage of the matter.

Rule 12.37(5) does exist. However, when the Judge checked the provision, its terms were materially different from the text replicated by Pinsent Masons. The provision dealt with an unrelated procedural point, and the quoted language appeared nowhere in the Rules at all. The Judge apprehended that a hallucination had occurred in Pinsent Masons’ research, and that a “cavalier attitude was being taken as to the accuracy of the material that it was putting before the Court”. Pinsent Masons was requested to provide a written explanation.

That explanation was given on 14 April 2026. In substance, Pinsent Masons apologised for the confusion but maintained that its letter had been intended only as a summary of several provisions and had not sought to replicate the text of r.12.37(5). The Judge found that explanation “impossible to accept”. The words had been set apart, italicised and introduced with a colon, in a manner akin to statutory style. An opportunity for candid correction therefore passed. Two days later, the authors of the 30 March and 14 April letters – including the senior associate and partner – were ordered to explain how the letters came to be written and whether AI had played any part in them. The purpose of the order was plainly to consider whether contempt proceedings should be brought against the lawyers involved.

The twist

The subsequent evidence revealed that both letters had been generated by the junior lawyer using the AI pilot program. The transcript between the junior lawyer and the chatbot ran to fifty-nine pages. The hallucinations started almost immediately. In addition to misstating r 12.37(5), the chatbot referred to irrelevant insolvency rules and provided misleading information about the Insolvency Act.

Yet, unreliable as its output plainly was, the chatbot itself exposed that unreliability. The transcript revealed it told the junior lawyer that it was:

“not fully confident… [it was] reproducing the exact statutory wording of Rule 12.37(5) with complete precision…for a submission to the court you should verify the exact wording against the current version of the Insolvency (England and Wales) Rules 2016 as published on legislation.gov.uk before relying on it. The last thing you want is to cite a provision to the court with inaccurate wording.”

When later pressed by the junior lawyer to identify where the “quotation” had come from, the chatbot said it comprised the junior lawyer’s own words from an earlier prompt. It again cautioned: “I would not want the Court to identify any discrepancy”, and that presenting a paraphrase as a direct quote “could undermine the credibility of the submission”. It further advised that, if the language could not be confirmed, the quotation marks should be removed, and the passage presented as a paraphrase. The junior lawyer only removed the quotation marks.

After the Court had exposed the hallucination, and the chatbot was eventually shown the correct r 12.37(5) in another prompt, it responded that a frank admission of error was warranted, that the assertions founded on the wrong text in the 30 March and 14 April letters should be withdrawn, and that an apology should be given to the Court. It also suggested seeking counsel’s advice on the operation of the rule. The junior lawyer relevantly typed: “I don’t think we should apologise – no”. Instead, the chatbot was instructed to construct a justification for citing the wrong text. That justification eventually became the 14 April letter.

Inadequate supervision

Although the Court focussed much of its attention on the conduct and judgment of the junior lawyer, the senior associate and partner also came in for criticism for failing adequately to supervise the junior lawyer and to check the relevant provisions.

The senior associate said she was “mortified” and explained that she had wrongly assumed the junior lawyer had correctly transcribed the relevant provisions. The evidence revealed that she enrolled in a course to better understand generative AI. Both the partner and the senior associate said they were unaware that AI had been used to draft the communications to the Court.

The Court concluded that, despite the misleading correspondence, there were mitigating circumstances that militated against referral. No one had set out to mislead the Court. The senior associate and partner might reasonably have expected a qualified lawyer, however junior, to present text in correspondence to the Court only from an authoritative source. They had otherwise behaved properly and in accordance with their duties as solicitors. Pinsent Masons had also been full and frank in providing the transcripts of the AI chat. It also referred itself to the SRA along with the senior associate and partner and agreed to pay its (now former) client’s additional costs arising from the incident. The Judge was satisfied that everyone at Pinsent Masons took this issue very seriously.

The Court considered the guidance given by the President of the King’s Bench Division in R (Ayinde) v London Borough of Haringey [2025] EWHC 1383 (Admin). Importantly, Judge Mullen noted:

“It is of course important that proper standards are enforced. There is a duty not to mislead the court in any event but there is a particular vulnerability in this case. These applications are uncontested. There is no opposing party to point out an error in correspondence sent to the court. The court is therefore particularly exposed to the risk of being misled if material is placed before it that has not been prepared with appropriate care and attention. The administration of justice cannot properly function if the court cannot trust its officers (in the case of solicitors) or those with an overriding duty to the court (such as barristers) to protect it from being misled.”

In a sentence that may be repeated in the future, the Court found that a lawyer “cannot outsource the process of legal research or of legal reasoning to an AI”. At its current level of maturity, Judge Mullen said that AI may be a “jumping off point for research and legal reasoning” but does not “do away with the need for proper research and thought on the part of legal professional, even a very junior legal professional”.

In the circumstances, Judge Mullen published his judgment as a public admonishment of the firm and the supervising lawyers, finding a prima facie case of breach of the duty not to mislead the court. However, he stopped short of referral. The SRA was better placed to investigate given the limited harm and the junior lawyers inexperience. As that the junior lawyer had revealed their identity to the SRA , no separate court referral was warranted.

It is worth pointing out that a frank admission and candour may have spared all concerned from public admonishment. Judge Mullen noted that:

… Had the true reason for the Purported Text been squarely set out and an appropriate apology offered, it may be that I would have concluded that, while regrettable, there had been no harm done and no further action would have been necessary beyond ensuring that Pinsent Masons’ clients’ costs in respect of that letter were met by the firm.

Conclusion

The case is a salutary reminder to the profession not to place blind faith in generative AI, even the more sophisticated enterprise-grade models adopted by international law firms. It also highlights the need within the hierarchy of a firm being appraised of how it intends to use its chosen AI service.

The danger is probably most acute where junior lawyers, perhaps eager to make a good impression, are reluctant to disclose their reliance on generative AI. Busy supervising lawyers and partners may simply not know that AI has been used, and may therefore be exposed to the kind of danger discussed in Cork v Smith.

In NSW, Bell CJ has said that it is of “cardinal importance” to recognise that qualified lawyers in NSW are admitted to legal practice and regulated by the Supreme Court. Responsibility for legal work remains personal and non-delegable; it cannot be outsourced to a computer program, however sophisticated.[2]

Given the proliferation of case hallucinations around Australia, and the capacity to drain court time and resources, courts can be expected to have an especially low tolerance with offending lawyers. That is particularly so when the court relies heavily on the profession - such as uncontested applications (as in Cork v Smith) or perhaps where the opposing party is self-represented. Referrals have already been made this year including by the Victorian Court of Appeal in Oberoi v Douglas [2026] VSCA 31 and by the Federal Circuit and Family Court of Australia in Pasuengos v Minister for Immigration and Citizenship (No 2) [2026] FedCFamC2G 96.

In the event that a court has correctly identified that material before it includes an AI hallucination, the response may determine whether the matter ends in a referral. After Cork v Smith, the following may need to be considered:

Notification: if not already done, notify supervising lawyers including the supervising partner and the firm’s risk or general counsel, and consider whether notification to the professional indemnity insurer is required.

Verification: check the true position immediately against authoritative primary sources. Do not ask the same AI tool to check its own work.

Inform: inform the client and address any potential costs consequences.

Correction: correct the record promptly, candidly and completely: withdraw the offending material and issue a full and frank apology to the court.

Preservation: preserve the full AI transcript and research trail. Be prepared to disclose it.

Advice: consider obtaining independent advice – including from counsel you’re your exposure to contempt, and on any further disclosure obligations to the court or the opposing party.

Self-referral: consider self-referral to the relevant regulator (in New South Wales, the Office of the Legal Services Commissioner; in England and Wales, the SRA or Bar Standards Board).

Remediation: review the firm’s AI use policy, mandate verification of all AI-generated citations and quotations against authoritative sources and require disclosure of AI use to supervising lawyers before any work product leaves the firm.

[1] Cork v Smith [2026] EWHC 1199 (Ch).

[2] A recording of the Chief Justice’s briefing is available via the NSW Supreme Court YouTube channel: <https://www.youtube.com/live/gpICy0lIxBA>.