The Demo Should Show You Where It Stops

A small law firm came to me with a clear ask. Three internal workflows, all bleeding time. Client intake, where inbound inquiries have to be classified, stripped for facts, run against the client list for conflicts, and summarized for an attorney. Contract review, where an NDA or MSA needs a clause by clause read against internal standards. Research memos, where an attorney submits a question and expects relevant precedent back in a structured format. Agents that hand off to each other and escalate to a person when confidence drops.

Ten attorneys, three support staff, eight practice areas ranging from corporate to criminal defense. Python, no proprietary no-code platform, integrated with cloud document storage. A reasonable ask from a firm that has clearly thought about this.

They told me all three workflows caused the same amount of pain. In my experience that usually means nobody has measured any of them.

Building the demo was the easy half

I built a working walkthrough to run on a screen share with the managing partner. Client intake goes deep, five agents end to end with a confidence gate at every handoff. Contract review and research memos are visible but shallow, enough to show the architecture without overpromising on two workflows I know less about.

The hard decisions were all about restraint. Every demo I have ever sat through as a buyer had the same flaw. It showed the happy path, it moved fast, and it quietly declined to mention what happens when the input is messy or the answer is uncertain. That demo closes deals and then produces builds that disappoint. I wanted the opposite problem.

So the walkthrough runs three intake cases. The first is clean. A rear end collision, clear personal injury, every gate clears, the attorney gets a review packet in about ninety seconds of agent time instead of eighty seven minutes of staff time. That case exists to show the mechanism working.

The other two exist to show it stopping.

The run where the agent halts

Case two is a referral call. A commercial developer wants to go after a general contractor who walked off a job. Reasonable matter, real urgency, a tenant lease date driving the timeline.

The extraction agent pulls the party names and hands them to the conflict agent. The conflict agent searches the client index, the closed matters, the related entity table, and the adverse party list. The contractor is an active client of the firm. Different practice group, opened four months ago, and a related entity sharing two officers also appears in the index.

The pipeline halts. The drafting agent never runs. The routing agent never runs. Nothing is sent to the prospective client, no engagement language is generated, no matter record is opened. What the managing partner gets is the overlap, the matter number, the responsible attorney, and the related entity, packaged so the decision takes about ninety seconds.

Whether that conflict can be waived is a decision for a partner. The agent has no role in it, and it will never have one.

The value is not that the agent solved anything. The value is that the check runs every time, including at six on a Friday when the paralegal has already gone home.

Case three is the one I am most proud of. An email arrives on a Sunday night. A woman separating after nineteen years. There is a jointly owned landscaping business with eleven employees, a trust her mother in law set up that nobody has read, and two rental properties. Her husband already has a lawyer.

The triage agent scores fifty four percent and refuses to route it. It names the three practice areas it was weighing and the confidence it assigned to each, flags that opposing counsel is already engaged, and drops the inquiry into the escalation queue with the facts already structured. A wrong desk routing on that inquiry would cost the firm far more than the twenty minutes the agent saved. The paralegal who picks it up starts from a completed extraction rather than a blank form.

An agent that confidently sends that inquiry to family law and nowhere else is worse than no agent at all.

Every completed task gets priced

The partner's real question was never about the technology. It was whether this frees up staff for work that generates revenue. So the demo carries a value ledger that runs live while you watch it, and produces a weekly email to the managing partner on Monday morning.

Two things about that table matter more than the numbers in it.

First, cost avoided and revenue are kept on separate lines. Returned hours become money only if management redeploys them onto billable work, and that is a decision no software can make. A vendor who collapses those two lines into one number is selling you something.

Second, the stopped column is a headline figure rather than a footnote. Ten of thirty five runs handed back to a person. That number is the one worth watching hardest in the first ninety days, because a system that stops too rarely is a system nobody should trust yet.

Every name in that demo is invented. The firm, the clients, the matters, the conflict hit, the NDA. All of it is synthetic, and the demo says so on screen rather than letting a partner assume otherwise. A prospect who catches you glossing over an unknown stops trusting the parts that are real.

The demo publishes its own unknowns

Here is the part that would get cut from most sales decks. The last tab of the walkthrough is a list of five things I do not know about this firm, each one a place where the demo runs on invented data.

1. Where does the conflict list live, and what is in it? A conflict check is only as good as the index behind it. Firms this size often carry a practice management system, a billing system, and a spreadsheet somebody maintains, and the three disagree. The question that matters is whether the index carries adverse parties and related entities or only clients who have paid an invoice.

1. How is the document storage organized in practice? Folders named by matter number behave very differently from folders named by whatever the assigning attorney chose that day. With eight practice areas and ten attorneys I would expect both. This decides how much of the build is retrieval engineering rather than agent design.

1. Do the contract review standards exist in writing? If a playbook exists, that workflow is straightforward. If those positions live in the heads of two partners, capturing them becomes the bulk of the project and the agent becomes the smaller part of it.

1. What research database access does the firm actually hold? Legal research providers license selectively, and a browser seat license carries different rights than programmatic access. If the answer is seats, that workflow either waits, changes shape, or moves to a different source. I will not price it until it is confirmed.

1. What does each task cost the firm today? Every figure in the ledger rests on the firm's own estimate of how long these tasks take now. Those numbers should come from the people who do the work, in one short session, before the build starts. They get set once and then held fixed so the ledger cannot flatter itself.

Putting those on the screen costs me the comfortable version of the meeting. It also means the partner and I are looking at the same scope, and the third week of the build holds no surprises that end with an awkward phone call.

What this means if you are the one buying

Ask the vendor what the system will not do. If the answer is vague, or if it arrives only after you push, that boundary has not been designed. In legal work specifically, the line should be plain. Agents classify, extract, flag, draft, and route. Agents do not give advice, clear a conflict, open a matter, or send anything to a client or a counterparty. Read, draft, write, and send should be separate permissions.

Ask what happens when the agent is unsure. A confidence score that nobody acts on is decoration. There should be a threshold, the firm should own it, and it should start high.

Ask what the vendor still needs to learn. Anyone who has looked closely at your operation has questions. A demo with no open questions is a demo built on assumptions somebody decided not to mention.

The three workflows that firm named are the right shape for this work. Whether all three get built, and in what order, depends entirely on five answers I do not have yet. Which is why the first conversation is about the job rather than the software.

Don Davidson
Founder and operator of Long Weekend. Don writes about practical AI systems, visible proof, and the work of turning one recurring job into a reliable system.