You'll know the same way you know with anyone on the front desk. It reports to you on the one number you already watch, like new patients on the schedule, and tells you where it got stuck. You read that report. Then you spot-check its work against things it didn't write: the schedule, the sent folder, the patient file.
You run the front of a dental practice. The phone rings, the schedule has holes in it by Thursday, and the new-patient forms come back half filled in. Somewhere in that, an AI employee has started answering the people who email or fill in the website form asking to book.
Call her Dana. Her job is new-patient coordinator: she answers everyone who writes in asking to book, sends the new-patient forms, and chases whatever's missing before the visit. She works in writing, in the inbox and the booking system. She doesn't touch the phone. That's still yours.
I give mine names too. It sounds like a small thing. It changes how you work with them, and I've written about why.
One thing Dana is not: a replacement for anyone on your front desk. The job she holds is the one that was getting done in the gaps, between the patient at the counter and the phone, usually by you, usually late. Taking it off the desk doesn't take anyone's place. It gives whoever's at the counter the room to look up at the person standing in front of them.
You can see what Dana does. Her replies are in the sent folder and her bookings are on the schedule. What those don't tell you is whether the job got done. Did everyone who wrote in get an answer? Who's waiting on something? Who wrote in on Saturday and never heard back? You'd have to read every email to know, and then you'd be doing the job yourself.
So the question isn't really how do I know it's working. It's how do I supervise someone I can't see, without reading everything she touches. You already know how. You do it with every new person at the front desk.
Your AI employee reports to you on the number you already watch
When I set up an AI employee, the first thing I settle with the practice is the one number its job is judged on. Not how many emails it sent. The thing you'd check anyway.
If its job is answering the people who ask to book, the number is new patients on the schedule. If its job is chasing the forms and insurance details before a visit, the number is patient files that are complete the day before. If its job is recall, the number is how many due patients wrote back.
Replies sent is not the number. Forty replies can go out over a weekend and nobody books. You know that from the front desk already. The phone rang all day and the schedule still has holes.
The report lands on a day you pick, and it opens with that number against what you expected. Then it tells you, with names, where it got stuck. A patient asking something about insurance it wasn't told how to answer. A form request that bounced. Anything it's waiting on you for sits at the top, so you deal with that first.
Stuck doesn't wait for the report. When Dana hits something she wasn't told how to handle, she flags it to you the moment it happens, the same way a new person at the desk would come and ask. The report is the week's summary. The flags come as they come, and you can open her work any day you want to, not only on report day. I do this with every one of my own. I pick the day for the report, I read it that day, and I answer the flags when they land.
You read the report. The dentist owns the practice.
An AI employee reports to one person on staff, and in a dental practice that's usually the person running the front. That's you. You read the report, you act on half of it, and you bring the rest to the dentist the way you'd bring anything else.
The dentist still owns what goes out under the practice's name, whether a person wrote it or an AI employee did. That's the same as it is now. Who answers when it gets something wrong is a separate question, and I've written about it in AI staffing is a hiring decision.
Reading the report is a small job. Open it on the day it lands. Read the number. Read the stuck list. Answer anything it's waiting on you for before you close it, because an AI employee waiting on your answer is a patient waiting on a reply.
What a report can still miss: the patient who got away
Say a man fills in the website form on a Saturday afternoon. Name, phone number, a few lines about a cracked tooth that's been bothering him all week. Dana replies that afternoon, which is what you'd want. The reply asks for his phone number so someone can call him.
He'd already given it. He reads that as nobody read his form, and he doesn't write back. On Tuesday he books with another dentist.
Monday's report said everyone who wrote in over the weekend got a reply, and that was true. You find out what happened later in the week, when you're going down the weekend list by phone and reach him. He's nice about it. The other practice called him back.
You write "lost" next to his name and dial the next one. You've done that before. The difference is that with the number set to new patients on the schedule, he shows up as a miss in Monday's report, while there's still time to call him. With the number set to replies sent, he never shows up at all.
That's the whole reason the number matters. Not for a dashboard. For Monday morning.
And for the rest of the week. With the weekend inbox handled, the person at the front desk isn't answering a form reply with one hand while a nervous patient waits at the counter. She can walk that patient to the chair. She can remember that his daughter was starting university. That's the part of the job only a person can do, and it's the part that sets one dental practice apart from the one down the street.
What you do next, once you've found a mistake like that, is in What to Look For When Choosing an AI Staffing Agency, along with the rule for when something comes to you instead of being guessed at.
You spot-check its work against things it didn't write
The report is Dana's account of the job. You still look at the job, a few pieces at a time, the way you'd look at a new hire's first week. I can't guarantee an AI employee's exact output any more than I could a new hire's. That's why someone checks.
Check it against records it didn't write. If the report says she booked someone, that person is on the schedule or they aren't. If it says she replied, the reply is in the sent folder, and you can read it the way the patient did. If it says a file is complete, the file is in the practice software and you can open it.
The checking fits inside your morning. Pick two of Dana's replies from the sent folder and read them as if you were the patient. Find two of her bookings on the schedule. Look up one name from the stuck list and see what happened to him. When something's off, say so, the same way you'd say it to anyone on the front desk. I read mine and I still send work back.
You'll check less as she earns it. You don't stop.
If you'd like to work out what your AI employee should report on, book a Let's Chat AI call. It's 30 minutes.