Today’s post is brought to you by the letters W, T, and F and inspired by this Spend Matters guest article by Matt Lhoumeau on The Last Contract Lawyer.

According to Matt, the legal profession is experiencing its iPhone moment because your competitors are closing deals in 26 seconds (and I certainly hope not!) using AI that outperforms human lawyers by 10% in accuracy (on what scale?!?). More specifically, he claims AI can complete a contract review in 26 seconds (spoiler: it can’t) while a human takes 92 minutes (on average I assume) and, furthermore, that this will cost you up to $6,900 (and this math makes no sense if the lawyer is only spending 92 minutes; because even top tier lawyers will generally only charge $500 per hour for a contract draft or review, so what’s the other $6,150 for).

Anyway, the most UGH! part of this article is not these false claims, it’s the missing information. Why is this the most UGH!? Because most of the claims the article makes are true, and when you tie all these claims together, if you don’t understand what this technology can’t do, and what risks it brings to the table (which is the missing information I refer to), you’re likely to believe the claims, join the AI religion, go all in on AI-CLM, and fire all your contract review lawyers. (And while I am no more fond of lawyers than the next guy, I am no less fond of them either, especially when they have a critical role to play.)

You see, the right AI engine (not ChatGPT) can:

• process a contract in an average of 26 seconds or less and perform a (very) large number of contract review tasks during that time
• cut approval times by 50%, and significantly reduce overall review times (that can easily add up to a calendar year for an organization that needs to review 500 contracts) to a small fraction of the time required (down to a few weeks to a few months)
• do more accurate pattern recognition than most humans, including “experts”
• significantly reduce outside counsel spend

And the benefits, when deployed properly, can be as great as the article claims. But this is the key — deployed properly. And there is no discussion of how you do that. The only piece of counter-information in the entire article is a reference to a Stanford Law School research study (that puts AI on Trial) that notes that AI tools using retrieval-augmented generation systems still hallucinate in 1 out of 6 benchmarking queries (but yet somehow outperform human reviewers on standard contracts? really?).

As we wrote earlier this year when we told you Don’t Kill All the Lawyers (and reminded you a couple of months later in our post that said you should embrace Legal tech … backed by lawyers), we’ve reached the point that you should (almost) never use a lawyer to:

• draft a contract
• review a contract for standard clauses, terms, and conditions
• locate the relevant statutes
• summarize your obligations
• summarize your incident response options
• etc.

because a tool can take your templates, standard terms and conditions, RFP, negotiation summary, and draft a better contract that most paralegals; ensure all of your standard terms and conditions are in there or review counter-party paper to ensure the same; review the redline you get (or are planning to give) that and determine which changes are good or indifferent for you; and then run the final contract through a standard agent for risk assessment to identify if the contract contains any known risks and flag anything that needs to be addressed, and do this better than a lawyer.

But what the tool absolutely, positively, can not do is:

• determine if the mitigations to known risks are sufficient in the particular instance addressed by the contract
• determine if there are any unique/non-standard risks that need to be addressed (that your existing checklists, templates, and review agents wouldn’t know about or check for)
• determine if there are any unique requirements for a contract with a supplier in a new jurisdiction that could require special considerations around key clause phrasing or standard risk mitigations
• have confidence beyond its models

You still need the human review, at least where it counts. And that’s the part you have to understand — and the part the referenced article doesn’t address at all.

If you’re a company doing a Billion dollars in business a year and signing over 10,000 contracts a year, you certainly don’t want to still be doing end-to-end manual reviews as that would be a minimum of 2 million minutes of review time, or the full time attention of almost 20 lawyers. Wasteful and completely unnecessary.

In fact, since you’re doing a Billion dollars or more (and likely 20 times that if your company is a Fortune 100),

• you probably don’t want to manually review any contract under a threshold (say $100,000) unless it is flagged as a high risk,
• you probably don’t want to spend more than an hour on a review of any contract under a larger threshold (say one million dollars) unless it is flagged as medium risk,
• you don’t want lawyers to read the remaining contracts end-to-end reviewing every clause and comparing those clauses against every checklist when it’s only the risks and unique requirements of the contract that require human intelligence

because limiting low value contracts to review only in high risk, low-mid value contracts to review only in mid-risks, and leaving the costly (but valuable) review time to the high-value or potentially high risk contracts will not only cut costs by 60% or more, but increase the value of the manual exercise.

Especially if those contracts are indexed by a natural language system that can allow the lawyer to ask key questions about the clauses that are in there, bring up the clauses she is interested in for a review, identify any processing flags, and apply her unique insights to the domain, jurisdiction, and business risks and ensure the contract accurately addresses all of these or focus her time on the right additions and modifications. For example, she might realize that the contract for on-site support in the nuclear power plant is extremely risky and the company’s across-the-board liability insurance requirement of 5 million is just not enough, realize that the AI safety requirements are not enforceable in the US and instead insist that the agreement be shifted to the Irish sub-entity and that jurisdiction apply, and so on. A check-the-box system won’t catch these things (as it can only look for risks it knows of and check boxes that have been identified), and neither will an open LLM (where you have no idea of the quality of the training, how much it is hallucinating, or, even worse, deliberately lying to you).

You still need a lawyer. Because, while it is an iPhone moment, it’s only an iPhone moment for lawyers who, if you aren’t using the tech, will be using the tech to help them focus on what’s important on the review stack and what isn’t. Because if the worst case is that you might lose an average of 10K to 50K here and there on every 100th contract in exchange for saving 10 Million on legal contract reviews and related matters (10 lawyers from outside council at an average of one million a year), that’s likely a worst case loss of a 2M loss in exchange for a 5X savings of 10M. And you know you won’t have many large losses because you’ll be able to focus legal review on the contracts that matter in dollar value or risk rating, not the contracts that don’t. And, all of a sudden, a close legal review of key contracts becomes a luxury you CAN afford!