What should a law firm do when an AI tool gets something wrong?

Treat it the same way you would treat a mistake made by anyone in the firm. Find out what the error was, how far it travelled, and what has been relied on because of it. Then correct it, tell whoever needs to know, and record what happened and why. The tool has no professional obligations; the fee earner and the firm do.

Most errors should never leave the building

The majority of AI errors a firm meets are caught at the checking stage, which is where they should be caught. A summary misses a document. A date is calculated from the wrong trigger. A point is stated with confidence and no source behind it. None of that is a firm-level incident. It is the ordinary friction of unchecked work, and it is why unchecked work never goes out.

So the first question when something goes wrong is not "what did the tool do?" but "why did our checking not catch it?" If the answer is that nobody checked, the problem is the workflow rather than the software. If the answer is that the fee earner checked and the error was still hard to see, that is worth understanding properly, because it tells you something about where the tool is weak and where your checks need to bite harder.

A useful distinction: an error found before the work is used is a near miss, and near misses are cheap information. An error found after the work has been sent, filed or relied on is an incident. Both are worth logging. Only one needs a response beyond the desk it happened on.

Steps to take when the work has already gone out

1. Stop the spread. Identify every place the flawed output has been used: letters, statements of case, advice notes, attendance notes, internal summaries other people are now working from. AI output copies quickly, and a wrong figure in a summary tends to reappear in three later documents.
2. Establish the true position from the papers. Do not correct one AI answer with another. Go back to the source: the document, the section, the correspondence. Write down what the position actually is.
3. Assess the consequence. Has a deadline been missed or miscalculated? Has the client acted on it? Has anything been sent to the other side, to a court or to a third party? Has money moved? The severity of the consequence, not the novelty of the cause, determines what happens next.
4. Escalate to a supervisor and to whoever holds risk and compliance. This should be the same route as any other file error. Do not create a separate, softer path for mistakes involving software.
5. Correct it and tell the people affected. If the client has been given something wrong, they need to be told, plainly, along with what you are doing about it. Firms are expected to be open with clients when something has gone wrong and to try to put it right. Delay tends to make an ordinary error into a complaint.
6. Consider whether it needs to be reported. Your insurance arrangements will have terms about notifying circumstances that might give rise to a claim, and there are regulatory expectations about reporting serious matters to the Solicitors Regulation Authority. Read those requirements as written and take the decision on their terms, not on a general feeling about how bad it was.
7. Write it down. A short, factual note on the file: what happened, when it was found, what was checked, what was corrected, who was told.

Do not treat the tool as the culprit

It is tempting to write "the AI got it wrong" on the file and move on. That record is useless six months later and it will not read well to a client, an insurer or a regulator. The honest version is more specific: what was asked, what came back, what was checked, what was not, and where the gap was.

That specificity matters because the answer usually points somewhere fixable. Perhaps the tool was asked a question the file could not answer and produced a plausible-sounding answer anyway. Perhaps the fee earner asked about a matter and the tool only had part of the papers. Perhaps the output named a source and nobody opened it. Each of those has a different fix, and only one of them is about the software.

Look for the pattern, not just the incident

A single error tells you very little. Three of the same kind tell you a great deal. Keep a simple log, even a shared spreadsheet: date, matter type, what the tool was used for, what went wrong, whether it was caught before use. Review it quarterly.

What you are looking for is repetition. If mistakes cluster around one task, stop using AI for that task until you can explain why it is going wrong. If they cluster around one person, that is a supervision and training question. If they cluster around one jurisdiction or one narrow area of law, that is a coverage question to put to the supplier directly.

Ask suppliers, before you commit, how their tool behaves when it does not know something. A tool that says nothing supports a point is easier to work with than one that fills the gap. Ask what happens when a document cannot be read, and whether that is flagged or silently passed over. The answers tell you how much of your checking effort will go on looking for absences rather than errors.

Where Alesis fits

Alesis is built to make gaps visible rather than fill them. When it cannot find support for a point it says what is missing instead of guessing, and if the papers on a matter do not answer a question it says so. Every point names its source, documents are read page by page so citations point at pages, and any page it could not read is flagged rather than skipped. It prepares drafts for a qualified person to review and sign off: it assists qualified professionals and does not replace them.