UK: ‘AI paralegal’ passes Solicitors Qualifying Exam with flying colours

An ‘AI paralegal’ called "Lawrence" has aced the British Solicitors Qualifying Exam (SQE), demonstrating his functioning legal knowledge is in line with the regulator’s expectations of junior solicitors.

‘Lawrence’ scored 74%, as against what the Solicitors Regulation Authority (SRA) said was a likely pass mark of 55-60% – and the most recent results show was actually lower, at 53%.

He was created by Lawhive, a consumer law platform that works with both freelance solicitors through its own alternative business structure, Lawhive Legal, and a network of law firms.

It delivers qualified leads, onboards clients, enables solicitors to work online, and provides secure file-sharing and messaging, invoicing and billing. Last year the firm raised £1.8m in seed funding.

It tasked Lawrence with answering the 90 sample questions on the SRA website for SQE1, a multiple-choice exam which assesses functioning legal knowledge. 

“With questions varying from contract law to criminal litigation, it was clear that Lawrence was able to learn, digest and offer considered responses to various legal situations,” said Lawhive co-founder and chief data scientist Flinn Dolman.

While there was no clear themes between the questions Lawrence struggled with, “it was apparent that the questions he didn’t pass were more complicated regarding complex chains of logic and wider context”.

Lawrence also had difficulties when two concepts shared similarities, confusing public nuisance versus private nuisance, for example.

Lawhive went on to present a mock will and probate case to Lawrence and a solicitor to compare their tone, empathy and legal knowledge.

It said that whilst Lawrence “managed to steer the conversation with the client to gain the necessary information about the client’s late relative’s will and assets, the conversation remained largely transactional and of half the length of the human solicitor”.

Though feedback from the client was positive to both, Lawrence was critiqued “for not showing as much empathy as the human counterpart”.

Lawrence also failed to question the client on their late relative’s spending habits and “asked around a wider context that ultimately uncovered financial liabilities the solicitor would need to be aware of”.

Mr Dolman said the experiments “clearly demonstrate there’s still no substitute for a qualified legal professional, especially in consumer law, when it comes to replicating the depth of human connection crucial in emotionally charged legal situations”.

He said the potential for Lawrence and his future iterations was as “a sort of ‘co-pilot’, providing support to solicitors on monotonous or repetitive tasks, freeing them up to focus more on the work that they are uniquely placed to do”.

Lawrence is currently being trialed with clients to test responses and whether it can assist with extracting specific information from them.

He is also performing monotonous tasks that are then looked over by a solicitor, such as filling in forms and drafting templates, as well as analysing documents provided by clients and answering solicitors’ questions, such as ‘When did Mrs Robinson first contact my client about the hedge?’