Every vendor will sell you an AI tool. Nobody owns whether it fits your process, who checks its output, or who is accountable when it’s wrong. I know what that gap looks like, because I’ve spent a decade working in it.
Sit through any AI demo this year and you’ll hear the same thing.
Look what it can find. Look what it can write. Look what it can summarize, predict, route, draft, answer.
All of it is true. None of it is the decision.
The decision is what the tool is allowed to touch, what has to be true before its output leaves the room, and whose name is on the result when it’s wrong. Those are questions about constraint, and almost nobody selling the capability is in the room when they get asked.
The seam nobody owns
Here is the pattern I keep finding inside large, regulated organizations.
The organization has the data. The destination exists. But the lane between them doesn’t.
Every vendor paves their own lane. Nobody owns the seam where they meet.
That’s not a figure of speech I picked for the sound of it. In road paving, the seam is the joint where two passes of asphalt meet, and it’s the most common place a road fails, because it’s the edge of two crews’ work and neither crew owns it.
An AI tool is the newest lane. It gets paved beautifully. And on either side of it sit the same unowned seams: the data that feeds it, the process it’s supposed to fit into, the people who have to act on what it produces, and the record of who decided what.
That’s why I don’t treat AI as a new service line. It’s the same problem I’ve been working on for ten years, with a model as the substrate instead of a database or an admin interface.
What a finding requires
I’m deliberately not going to list what AI can find in your data. In healthcare especially, a list like that gets skimmed by a compliance officer who reads the nouns and skips the caveats, and it’s the nouns that get an organization in trouble.
So here is the other list. Before any output from an AI tool becomes a decision, someone in your organization needs to be able to answer six questions about it.
- Where did the data come from? Which system, which extract, which date. If two systems disagree, which one the model was told to believe.
- Is there permission to use it for this purpose? Data collected for one reason is not automatically available for another. The law attaches to the data, not to the platform it’s sitting in.
- What information actually enters the model? Not “does the vendor say it’s fine.” Which fields travel into the prompt, the index or the training set, and where they go after that. This is the question I help you assemble the answer to, field by field. Whether a given field is permitted to make that trip under a given law is a determination that sits with your counsel, and I work through it with them.
- What happens when it’s wrong? Every model is wrong some of the time. The question is whether a wrong answer is caught before it reaches a patient, a customer, a regulator or a balance sheet, and what the correction path is.
- Where does human review sit? Not “a human is in the loop,” which everyone says. Which human, at which step, with what authority to stop it, and what evidence that the step actually ran.
- Who owns the resulting decision? A name and a role. If the answer is “the tool” or “the vendor,” the seam is unowned, and it will fail there.
Counsel can’t rule on a data flow nobody has written down. In my experience that’s where the third question stalls: not on the law, on the fact that nobody has assembled the list of what actually enters the tool. That gap isn’t a technology problem, and it isn’t a legal one yet. It’s an ownership problem, and it’s the same ownership problem that leaves a location finder showing an address that closed two years ago.
I’ve closed this seam before, without a model in sight
Years ago, at a large healthcare and life-sciences enterprise, I built the system that managed the online test menu, the catalog of services a customer could look up before they ever walked into a location.
The interesting part wasn’t the build. It was who owned it afterward.
Operations owned the process. So I built the administrative interface to their requirements, kept their audit steps intact, and left their final approval exactly where it already lived. My job was to build the thing that made the seam ownable, and hand it to the team whose process it was.
I didn’t become the new dependency. The catalog didn’t need me to stay accurate. That’s the whole test.
An AI workflow is the same move. Objectives, inputs, stages, dependencies, rules, decision points, expected outputs, and the human review step that the workflow is not allowed to skip. Build it the way the owning team needs it, with their approvals intact, and hand it over.
I can show you the loop, because I run one
I’ll say the quiet part. I use AI to run this practice, and I’m open about how on my About page.
Here is the part the demos leave out. AI doesn’t know what I know. Every insight, every fact, every rule it works from in this practice, I had to provide. It can’t do any of that on its own. To use it effectively, you have to already know what the correct output looks like, or you can’t tell when it has handed you the wrong one.
So what I don’t do is let it decide.
Every piece of content that goes onto this site passes through a pipeline with a gate in it. The gate checks accessibility, brand voice, claims accuracy and privacy compliance, and it requires my explicit approval of a proof before anything is built. I wrote the rule so that the workflow refuses to skip that step, including when I’m the one in a hurry and tempted to skip it.
There’s a second rule that stops two AI sessions from quietly overwriting each other’s edits to the same live page. There’s a third that blocks a document from shipping without a version stamp and a matching register entry, so a file that comes back attached to someone’s email three months from now can be identified.
None of that is impressive technology. It’s constraint, written down, enforced by the workflow instead of by memory. Every consultant can say “human in the loop.” I built the loop, and I can show you where it sits.
Where I stop
I don’t build models and I don’t run them. I’m not going to recommend one vendor’s tool over another’s. And I’m not an attorney: whether a specific use of your data is permitted under a specific law is your counsel’s determination to make. My part is to make sure counsel is ruling on the real data flow, not a vendor’s summary of it: I assemble the data types, trace where each one travels, and work through the questions with them.
The rest is the layer above the build. Which data is in scope and which is not. Where the review step lives and who holds it. What the audit trail has to prove. Who owns the decision once the tool has made its suggestion. The same discipline I’ve applied to platforms, integrations and front doors for a decade, applied to the newest lane on the road.
Everyone is selling what AI can do. I’d rather help you decide where it stops.
If you’re deploying AI right now and you couldn’t hand counsel the answer to question three this afternoon, that’s the conversation worth having first.
This post describes a governance method, not a compliance determination or legal advice. Whether a specific use of data is permitted under HIPAA, state privacy law or your own contracts is a question for your counsel.
Related: It’s Not Skynet: The Truth About How AI Actually Uses Healthcare Data — what the models actually do with the data. This post is about who governs it.
