Skip to content
Alesis
Start free
All posts

Blog

What should we do when AI says it cannot find the answer?

Treat it as information, not a failure. Work out whether the gap is in the papers, in the sources or in the question, then do that part of the work yourself and record what you found.

Alesis · · 5 min read

Unwritten notebook pages
Photo by Markus Spiske on Unsplash

A tool that says "the papers do not say" or "I cannot find support for this" has told you something useful. The right response is to work out what kind of gap you are looking at, deal with that gap by ordinary means, and note on the file what you did. The wrong response is to rephrase the question until something that looks like an answer comes back.

Why a refusal is worth more than a confident guess

The risk everyone worries about with AI in legal work is the fluent answer that is not supported by anything. An answer that names its source can be checked in a minute. An answer that invents one costs you an hour and your credibility.

So a tool that stops short is behaving the way you would want a trainee to behave. If a paralegal told you the completion statement is not in the bundle, you would not tell them to try again more confidently. You would ask where it might be, or get it from the other side.

The habit worth building across the firm is this: a refusal is a finding. It goes in your note. It may even go in your next letter to the client or the other side.

Work out which gap you are looking at

There are four common reasons an answer does not come back, and the response to each is different.

The papers do not contain it. You asked what the service charge cap is and the lease schedule was never scanned. Or you asked when the claimant first reported the injury and the GP records stop two years short. This is a disclosure or collection problem, not a technology problem.

The page could not be read. A scan is blank, a page is upside down, a fax header has eaten the first two lines, a photograph of a handwritten note is too dark. A good tool flags the page rather than quietly skipping it. Go and look at that page yourself.

The official source does not say it plainly. Some questions are not answered by any statute, rule or published guidance. They are answered by judgement, by the way a particular tribunal tends to approach things, or by a line of authority that has to be read and weighed. No tool should pretend otherwise, and you should be wary of one that does.

The question was too wide or too vague. "Is this claim any good?" is not a question with a source behind it. "What does the contract say about termination for convenience, and on what page?" is.

Most refusals in practice are the first two. They are problems with the file, and finding them early is worth money.

What to do next, in order

  1. Read the reason given. A gap in the papers, an unreadable page and an absence of authority are three different problems and the tool should tell you which it is.
  2. If it is a papers gap, find out whether the document exists. Check the file index, the case management system, the email archive, the client's own records. Then decide whether to chase it, request it formally or proceed without it and say so.
  3. If it is an unreadable page, open the page. Rescan it, get a clean copy, or type out the relevant line yourself and put that in front of the tool.
  4. If it is an absence of authority, do the legal work the ordinary way: the legislation, the practitioner text, the authorities, counsel if the point warrants it. Do not treat silence from a tool as a conclusion that the law is silent.
  5. If the question was loose, narrow it. Name the document, name the clause, name the date range, name the person. Ask for the page.
  6. Only then consider asking again. Asking the same question five different ways in the hope of a different answer is the behaviour that gets firms into trouble.

Record what the gap told you

A missing document found in week two is a chase letter. The same document found the week before a hearing is a problem. So write it down at the point you find it.

A short note works: what you asked, what came back, what you did about it. If you chased a document and it never arrived, that note is the record of why your advice was given on the material available. If you decided the point needed counsel, the note shows when and why.

This is also how a supervisor can tell whether AI is being used sensibly on a file. A file where the tool was pushed until it produced something looks very different from a file where three gaps were identified and two were closed.

Build the expectation into training

Tell people at the outset that a refusal is a normal output, not a sign the tool is broken or that they have asked badly. Firms that skip this step end up with fee earners who quietly keep rewording questions because they think an empty answer reflects on them.

It also changes what you look for when you assess a tool. In a trial, ask it about something you know is not in the papers. Ask it for authority on a point you know is unsettled. How it behaves when it has nothing tells you more than how it behaves when the answer is easy.

Where Alesis fits

Alesis answers questions about a matter from the matter's own papers and names the page each answer came from; if the papers do not say, it says so. When it cannot find support for a point, it says what is missing instead of guessing, and any page it could not read is flagged rather than skipped. It assists qualified professionals and does not replace them, and it does not provide legal advice.

Alesis assists qualified professionals and does not replace them; nothing here is legal advice. If a point above is wrong or out of date, write to us and we will correct it in writing.

Put a real matter to it

Create an account, open a matter and upload the papers. Every point it makes names where it came from.

Get started