Recently had a deeper look at the capability of an AI platform

For legal services. Specifically investigations, document review, litigation, regulatory supervision and enforcement processes, and the market’s current and projected use. But the following applies to transactions too. 

the time has come to accept that the human mind and our ability to manage lots of them to deliver an hours-worked-based product is at a point of obsolescence. It’s not just the administration of tedious volume activity - the insight and intelligence is there. The joining of dots and analysis. 

The speed and quality, breadth of capability (across unstructured data wherever it is in your organisation or client’s) outstrips what a legal team can deliver. Yes some senior and experienced mind may provide a gloss or recommendation on the work product - partner to client board or senior associate to client senior lawyer - but the justification for use of people to crawl over documents and dredge up the evidence is dead.  Dead. Unjustifiable. 

We used to be able to say ‘oh oh halucinations and error rate’ but now we must accept that it is not just as good as your best associate, trainee, paralegal - it is better.  Way better. Faster, more thorough and more agile. Horizons the human mind cannot cope with over weeks and months without burnout or failure, covered off in seconds. The change in the last 18 months has been staggering. 

If you are a lawyer or firm whose business model simply churns our bills on the basis of aggregated team hours the presumption should be that there is no ethical justification for your approach. You are fleecing the client for 90%+ of the work product. 

Fine, rebut that presumption if you can but the client doesn’t have to wear this. They wont.  If you aren’t rethinking your value add proposition yesterday then the clock is ticking down to your insolvency. 

This is going to go through the market like a sodium metabisulphate dose before an endoscopy in the next 24 months. Be very concerned and be agile. 

I warn you not to be young. I warn you not to be led by dinosaurs. If you are, move now to an enlightened and committed business which actually wants to tread new ground and offer a different proposition where the lawyer is the strategic consultant - a specialist and highly qualified business consultant with a legal fine-tune. Not some fossilised firm that says ‘we are pivoting’ and thinks this is about the internet and that lawyers will always be in demand. 

And if you are older, mid ranking and half established and think the ground is safe it is not. If your firm thinks it can deliver with a flow of junior lawyers learning their trade on the job and senior ones owning the equity and charming the birds with top-up wisdom then have a word with yourself. The junior flow will dry up. The flow to mid and senior will not work right. The old structure will die on its arse. There will be a dearth of this elusive ‘experience’ that justifies the old model. 

The fixed fee value proposition with AI driven cost and quality drivers will rip it all up. 

Fine, if you are a Bottomry Silk at 7 Old Foreskin Chambers, carry on, you’ll be dead before the Bar thinks ‘hang on a blithering minute’.  The courts and judiciary will embrace it and outpace you. 

But if you are in a law firm and it’s not dramatically shifting the way it does what it does, beyond recognition, then you’re mad to stay there. 

Heh. I have just posted a request for products like this. Are there any?

“And if you are older, mid ranking and half established and think the ground is safe it is not. If your firm thinks it can deliver with a flow of junior lawyers learning their trade on the job and senior ones owning the equity and charming the birds with top-up wisdom then have a word with yourself. The junior flow will dry up. The flow to mid and senior will not work right. The old structure will die on its arse. There will be a dearth of this elusive ‘experience’ that justifies the old model. 

The fixed fee value proposition with AI driven cost and quality drivers will rip it all up.”

This bit - we are already there.

I think you sum up the point Heff. As it stands you can let e.g. Claude take over your laptop. So if you had e.g. 100 SPAs saved on there you could quickly have it interrogating those for precedents, searching the internet for party details, populating ancillary documents etc. 

The issue comes from confidentiality/client requirements. So at the moment either a) firms are breaking it (up to at least mid-market this is happening whether management know it or not or b) you are having to use e.g. AI-enhanced products like relativity or Harvey and carry the raw materials to it. 

The reality is all law firms have largely been taking the piss out of clients for years through failing to structure data and comprehensively capture it, and implement standard project management techniques. Their moat has been that alongside jokers like financial advisers the legal fees are still fook all. 

The main area for me where there seems to be no hope is regulatory advice. I just can't see how you protect this e.g. every employment dispute must have been had, the source material (i.e. legislation is all out in the open) and if someone can aggregate the closed off interpretation i.e. each decent law firm's advice on it, then there is fook all point in paying through the nose for it even now. At least transactional work still has the "we want it done by Friday" factor applied to a load of disorganised shit that mean it is protected until clients work out the same arguments have been run countless times under the cloak of each law firms' systems, and demand it is exposed so you click the options on a term sheet and a computer with perhaps 1 person vetting it plugs exactly the correct terms into the agreement. 

thanks, BM, that's my experience too. 

Posting my OP from the thread I started at the same time, let's amalgamate them here:

Heffalump 16 Apr 26 09:22

Reply |

Report

I'm aware of the abilities and defects of current AI tools in relation to what the law is. This query is about AI products for lawyers to analyse factual docs by way of case preparation. Is anyone using anything of use which complies with UK data protection and client confidentiality rules applying to solicitors and bazzas?

But if you are in a law firm and it’s not dramatically shifting the way it does what it does, beyond recognition, then you’re mad to stay there. 

So, pretty much every large law firm in the City and every major US city?
 

Having recently been designated our firm’s lead on AI implementation (in my practice area) I have a similar impression to mutts

The old model of expensive humans expensively sweating and gurning to nobody’s great benefit is dead, and in the round that’s a good thing. We are on the cusp of a massive leap forward in productivity - the question is who will get on board and ride it.

Dang - even the bottomry juniors at 7 Old Foreskins in Row are on the move! 

The fact Laz is put in charge of it sums up law firms. See also LinkedIn where cashed-up 50+ yo knowledge fogeys with no tech ability at all w**k on about AI when there jobs have been checking for typos in standard docs and delivering training on stuff that has fook all relevance to modern lawyers but ticks a box. 

There are probably 5 law firms that "get it" and are burning the cash to get there. All others are re-arranging deckchairs and will be the UK 10-30 ranked accountants of 30 years ago. 

AI replacing lawyers would be hugely positive.

All the time and money that the country spends on legal docs/ interpretation is a drag on economically productive activity.

The whole thing should be taken over by AI and the sooner the better. Bright grads will then abandon overpaid legal roles and do something more useful, like trying to find cures for diseases, or developing some new tech.

Checking for typos in documents, while it hasn’t actually been my job for 25yrs, is good background for a job that involves implementing technology in a business 50% of which still boils down to producing accurate documentation.

I’m not in charge of the tech itself anyway, I’m “in charge” of getting it to produce what needs to be produced in a legal business. There are people who know more about developing software, but this is no longer about developing software. The software exists. This is is about using it. I’m King User, and unlike you Sun I’m still under 50.

As for the products, I think Legora is probably what you are after Heff, though every provider of DD tools now has an AI overlay that is compliant with client requirements e.g. Relativity, Harvey, Kira.  

"King User" - the Midas touch, but not in a good way. 

Whatever sun. Anyway it’s fair to say my long term ambitions are more about owing a proprietary LLM than owning 1/500th of a law firm.

Which are the firms that get it nana? Or are you speaking generally?

Tangentially, the number of people in business organisations, as widely defined, who literally cannot express themselves in writing whether it be to give instructions or understand advice is truly staggering. 

And yet I see so many people who are paid what ordinary folk could retire on. 

None of this will end well, whether in or out of the workplace. 

CC, Latham, Bakers, Simpson Thacher plus a few others. There's pretty much an inverse correlation between firms banging on on LinkedIn and elsewhere about their tech prowess and their actual investment and its quality. There are several firms that have spent millions on AI tools and routinely bang on about their use for BD purposes, but it is mostly lawyers using 3 line one off prompts to do marginal stuff like doc comparison. It's like getting the Cambridge Physics PhD vac student to proof read some board minutes. 

We are just about to trial Legora and have also had a client ask us to confirm in a pitch for review large number of documents that we'll use AI for that.

This is a great example of law firms' complacency though. They could have been doing tech driven transactional doc review for 10+ years with a couple of data scientists. They are only offering to do it for clients now they are "woke" to the lawyer scams. 

Thankfully mortgage lenders still haven't accepted electronic signature so I've got a few more years before they'll meaningfully embrace more technology.

"We used to be able to say ‘oh oh halucinations and error rate’"

Actually no. We still say that because that is how machine works. Errors are an intrinsic part of model fitting, they can be tweaked at the expense of other things but they cannot be removed. 

Overfitting and underfitting in machine learning | SuperAnnotate

The Australians seem to be onto this point - https://www.fedcourt.gov.au/law-and-practice/practice-documents/practice-notes/gpn-ai

"2. Court’s expectations

2.1 The Court’s expectations concerning the use of Generative AI in connection with proceedings are as follows.

   (a) Any person who uses Generative AI will have a basic understanding of its capabilities, and its limitations and risks.

   (b) Any use of Generative AI must not adversely affect the administration of justice. This requires users of Generative AI to be guided by, and act in accordance with, their existing legal and professional responsibilities.[3] It follows that there will be circumstances where it is inappropriate to use Generative AI at all or where users ought to be transparent about its use.

   (c) If the Court requires it, a person must disclose to the Court if (and how) Generative AI has been used in a proceeding."

4. Specific instances where caution is necessary...


Pleadings, written submissions, lists of documents and other documents or information lodged with or sent to the Court...

etc, etc"

I have recently had the misfortune to review numerous reports prepared by another firm where they have clearly used AI and the output is full of guff. 

I can see how time can be saved on the actual grunt work of duplicating and summarising text, but a human who knows what they are doing needs to review the output and make it fit for purpose. IME. 

 

Can someone please tell my why it is a 'scam' for anyone to charge what the sophisticated business client/customer will pay for a service? 

 

Most law jobs are going to be lost, or significantly changed with a large reduction in the fees that can be charged. Intellect and experience are going to become readily accessible and very cheap to obtain, where before they were rare and expensive. A few will thrive but they will be the ones that can adapt quickly enough.  Your average firm with a bunch of mid-40s/50s equity partners is in for a shock.  

I agree it is coming but not as fast as some people think. From what I have seen there are still many absolute howler errors/hallucinations in the work product and accuracy comparisons to humans are misleading. Night equals day reversals of positions are common in a way they just aren’t in human produced stuff (in my experience at least). 

 You also have the issue of data to train the models on and that will be harder to access in regulatory compliant ways than people think. Lawyers will be much slower to be willing to ask for forgiveness/just not give a sh1t than tech companies.  Where there are vast publicly available data sets progress will be quicker. 

So I would say the really big tipping point is probably closer to 10 years away than two. For city law anyway.  Lower down the food chain someone might come and massively disrupt sooner.  

We have a good regulatory moat so will be protected from ai-apocalypse for a while imo. Only humans can give legal advice, so until that changes we’re safe from our Anthropic overlords. I think it will make the profession more productive and richer ;)

Only humans can give legal advice

 

you see, this is where all the trouble is.  The profession has decided that all sorts of junk constitutes legal advice and charge for it. The reality is a heck of a lot of what lawyers now sell as "advice" is not advice but analysis and re-ordered review of material. AI does that brilliantly. 

Agree but JP Morgan bankers aren’t gonna be prompting Claude directly for a while to draft their back leverage documents 

This is not about Claude or similar.   

Claude: according to their own data as at 2026 the main users of Claude are software developers, enterprise technology firms, and knowledge workers who require advanced coding capabilities, large context windows, and sophisticated data analysis. 

The main user of Microsoft Copilot are enterprise knowledge workers, professional services firms, and large organizations using for informal purposes (e.g. refining work product - using like an editorial service or first draft producer) and young users relying on it for homework and undergraduate study support.

By contrast Legora and simiar are tailored to legal professionals within large law firms and corporate legal departments. It is relatively emergent -  800 customers and tens of thousands of daily users as of March 2026. The platform is primarily used for AI-powered legal research, document drafting, review, and collaboration on complex matters. It's built to support the legal activity we are all very familiar with. It is not the same as copilot and claude etc. 

Banks' legal departments have always been huge spenders. In my day in-house I ran a £750m budget on litigation alone. Much of the costs I incurred were process around discovery and doc management. That part of an investigation or litigation just sucks up the hourly rate.   None of that relates to the other side's costs for cases lost, which was a separate budget so the net number was even bigger (plus the headcount cost of in-housers).  The finance guys were chewing our arses constantly and any GC who can procure a saving like this will be forced to, by a CFO.  Banks are probably the most likely to mandate this. 

I should think JPM, Goldmans etc would be among the first to go in properly large on this, contrary to your views. 

Maybe "scam" is a bit strong, but in the end no different to the teeth sucking builder that overcharges pensioners. In the end big data will show up the complexity of what you really do and while you were making your fat margins someone else will have got their ducks in a row and gone for value and scale as the driver of overall profit. 

Lawyers at the moment are protected by the "guild" factor. The tipping point is when a handful of firms pay to get it right, as at that point they likely don't need to hire any more staff and can come for work at their level and lower and lower down the foodchain. In my view, whining about hallucination is akin to most English firms' attempts at outsourcing/paralegalling - they will not pay/invest to design and implement a system that has guardrails and accepts imperfections as a bug that needs managed rather than removed or "forget it I'll do it myself". It's the graduate/vac schemer that's told to e.g. draft a legal opinion and then ignored because it didn't come back 100% perfect. 

Large banks from a transactional perspective will wake up to marshalling their available data, and if they are clever will co-opt their closest advisers to give up theirs too to assist them. From that point is is easy to construct a database that contains all variations of virtually all permutations of a work product, and then it is only a bit more work to basically have the model wording for every potentially agreed scenario. Effectively you have an uber-ISDA for every work type and negotiation, if there is any, comes around the margins. 

At some point some bank will also ask for e.g. pan-European law firms to document all feasible corporate authorisation methods, registration requirements etc and will be able to have a good first go at all necessary ancillary transaction documents, which might be sent for checking to exploit a law firm's insurance policy. 

Anyone seen the legora ads with Jude Law? 

Very funny and worth a watch.

Muttley what about judges?  Do you think they will be affected?  If so, how will they be affected?  In some areas like family financial remedy cases they have wide discretion which is very difficult to appeal against which I think is a big problem.  

I’ve always thought about this in terms of dividing up each £ charged between low cost overhead admin, lower end analysis, higher end actual advice and at the top actual judgment calls. 

Solicitors have always been able to max out the % in each £ billed of admin and lower end stuff, for which obvs they pay employees as little as possible, and minimise the actual judgment calls, which are risky. Obvs they lie about how they do this, not least to each other. 

The spread of tech and now AI means, very simply, that 95% of each chargeable £ disappears. 

The only firms who deserve to survive are those with actual expertise and low overheads. And god help anyone trying to train the next generation, most of whom can barely string a sentence of their own together, let alone write one. Because they know it’s all about owning the means of production not knowing how anything works. 

This is all a recipe for bloody revolution well beyond the 0.1% fvcking up and finding out. 

Mutters - technical point, Legora is based on Claude (Anthropic). They don’t have their own ai model, they white label Claude and their business model is about confidentiality mainly - if you prompt Claude your info goes into the model, if you prompt Legora is stays within a closed system for the relevant firm. Harvey is identical, but uses ChatGPT for their LLM. There’s been a lot of talk about whether both are in danger of being replaced directly by Claude as Anthropic have released legal co-working tools themselves. 

There’s going to be a human giving legal advice ultimately unless they change the law (which we might protest about!). Maybe the banks will insource everything to in-house legal with ai, but my feeling is that they’d prefer to outsource still to law firms for risk purposes 

Every time I see your name it brightens my day a bit

 

My lyrics are bottomless!

The elite US shops won’t adopt it unless it makes them money 

I’ve used it a lot.  An awful lot.  And found it was incredibly useful for certain tasks.  The hallucination thing – yeah, that happens but a lot more on the free variants.  

On our captive (paid for) version, I’ve found it to be nearly brilliant.  

But with a massive caveat – the wrong step problem.  It works by reading and predicting and then creating a narrative (you can ask it for bullets, or prose or a mix) and it’s great.  But it figures out what is the next word in a string, and then the next one. Etc.  The greater the ask, the more words it respond.  And where it makes a mistake – it’s not got the human bit of where it say “oh, that’s not right – start that bit again”.  It keeps going.  It doesn’t stop – it will continue in the wrong direction until it reaches the end which is not where you wanted it to go at all.  

I’ve given it a doc and asked it to analyse on the basis of 10 questions.  Questions 1-4, perfect.  Then a wobble at Q5, and then the wheels come off entirely.  It was telling my things that the document didn’t contain – adding in scenarios not analysed in the source material.

I’ve also found that you need to ask the same question via a “clean” prompt 2-3 times.  It takes a different route each time. All valuable, but the first response requires a careful testing and probing.  And that the problem.  I can dismiss the errors, and I can “catch-it” the same way I can catch a junior going off in the wrong direction.  I learned that through making mistakes.  I’ll retire in 15 years or so.  The generation after will never have had the fear of trying to patch-up a mega ffck up that makes you better next time. So tomorrow’s senior lawyer wont have  scooby of how to be good at what they do.  

That advert for Copilot (I think) where two people negotiate across a table (with chairs getting taller and taller as they establish their positions) is truly chilling.   Is the future set that every problem gets fed into an AI and then all parties accept what it says?  Precogs and Minority Report.  Not for me.

I posted about this a few months ago and people scoffed. The technology is potentially some of the most impactful that humans have ever created but the least regulated.

AI experts suggest there is a realistic chance (tends to range between 5 and 30% depending on who you ask) that the tech will attack humanity, yet still no proper regulation. 

Can you imagine trying to open a nuclear power station and the odds were between 5 and 30% that it would blow up?!

So tomorrow’s senior lawyer wont have  scooby of how to be good at what they do

That has been the logical outcome of this stuff from the minute it appeared. Several years ago.

The odds of a nuclear power station blowing up are 50/50. It will, or it won't. 

I would say the entire issue is what a modern supervisor considers an adequate instruction/prompt. We have loads of juniors that at least claim they are given short email or verbal instructions to do/draft things they have never done before. An adequate "prompt" years ago would be sitting someone down, explaining the rationale for a transaction, the structure and the documentation on transactional work. That person would then sit in the room with you all the time, constantly back and forthing with you and with you listening to and checking what they are doing. There are all kinds of reasons why this is not feasible now, but it is very much a garbage in garbage out scenario that will dog AI adoption as much as it has dogged trying to get lawyers to delegate, use legal tech, apply project management, do a day's meaningful further education. The problem for many law firms is what was seen post-GFC - the organised top came for the middle and the organised middle came for the bottom because they look beyond the end of this financial year in meaningful ways (rather than creating a strategy slide deck), and realising this involves hard work and investment not waiting for the travelling salesperson to show up with the DoAllYourWorkfor£0andChargeTheClient£1,000,000 2000.