Research · Method · 17 September 2026

How do you test whether an AI agent can book a restaurant?

Notes from building the harness behind a 200-venue field study, and one finding that surprised us two months later.

The question sounds simple until you try to answer it. Can an AI assistant book a table at this restaurant? You cannot ask a model, because it will tell you what it believes rather than what it can do. You cannot read the website, because a page that talks about reservations is not the same as a page that accepts one. And you certainly cannot book two hundred tables to find out.

Here is how we ended up measuring it, what broke, and why the result got more interesting in September than it was in July.

The measurement is where you stop

The harness drives a real browser through a venue’s public booking path: find the entry point, open whatever widget or form is behind it, fill date, time and party size, and go as far as the flow allows.

Then it stops, at the exact moment the flow asks for the guest’s own details or offers a final confirmation.

That stopping point is the measurement, not a shortcoming of it.

“Reached the step where only the guest’s name and phone remained” is precisely what “an assistant could have finished this” means, and it costs the restaurant nothing. Booking two hundred phantom tables to produce a statistic would be a strange way to argue that software should be more considerate.

How to test an AI agent without booking a table: three barriers

We kept all three, because a study that accidentally creates real reservations is not a study, it is a nuisance.

The user agent names the study openly, with a contact address. A crawler that hides is a crawler that knows it is doing something it should not.

Judge from controls, never from prose

The first version of the harness believed too easily. A page that said “reservations” scored as a page with reservations. A restaurant describing its private dining room counted as a bookable flow.

So the confirmation check was rewritten to look only at real controls. It counts input fields whose name, id, placeholder, label or type match personal details, and it counts them only if the agent actually reached a booking surface first.

Text is evidence of intent. Fields are evidence of capability. Only the second is being measured.

The same rule applies to what stopped the agent. The harness records it by name:

When we re-ran the sample with a newer version that tells a consent banner apart from an anti-bot check, the totals moved. That is a change in the instrument, not a change in the world, and it has to be said every time the two runs are compared. Two numbers from two different instruments are not a trend.

Browser agent vs API: driving a page is not calling an interface

This is the distinction the whole study turns on, and it is easy to blur.

An agent driving a browser is pretending to be a person. It moves a virtual mouse, waits for scripts, guesses which box is a date picker. It works, sometimes, slowly, and it breaks whenever the layout changes.

An agent calling an interface does something else entirely: a request, a schema, a typed response. Availability comes back as data. A booking comes back with an identifier you can check later.

Both can end in a reservation. Only the second is reliable enough to build on, which is why the platforms assistants actually integrate with offer the second kind. So the study reports two separate numbers, and they answer different questions.

Is llms.txt worth it? What 163 websites did in two months

Two months after the first run we checked the same 163 domains for the files that make a business legible to software: an MCP manifest at /.well-known/mcp.json, an OpenAPI description, an AI plugin descriptor, an llms.txt.

By September one in five restaurants in the sample had a plain-text document explaining itself to a language model, up from nine sites in July. Hours, cuisine, atmosphere, the things you would tell a guest. And not one of them published a way for that model to reserve anything.

It is a good instinct aimed at the wrong layer.

llms.txt improves how accurately you are described. It does nothing for whether you can be used. An assistant that reads your file and cannot act on it will describe you correctly to the guest and then complete the booking somewhere else.

If you are building for this, the shape of the answer is not complicated. Something that answers three questions with data rather than pixels: what do you offer, when are you free, and can you take this reservation now. It can be an OpenAPI surface, an MCP server, or anything else with a schema on both ends. What matters is that a machine gets a typed answer instead of a screenshot.

AI agents and captchas: the uncomfortable middle

There is a kind of failure that is nobody’s fault, and it is bigger than we expected.

At 26 of the 200 the agent ran into the venue’s own protection: an anti-bot check, a captcha, a device check. Those were installed on somebody’s advice, for real reasons, and they do not tell a scraper apart from a guest’s assistant.

The right conclusion is not “turn off your captcha”. A consent banner is not security at all, since a scraper never renders it. A captcha is real security, but it is usually applied to the whole site, including the part that only reads opening hours and free tables. Those are public facts, given out over the phone all day. Protect the write path, where damage is possible. The read path answers a question any stranger could ask by calling.

How varied this is in practice: two venues on the same booking system, in the same run, gave opposite results. One stopped the agent at a cookie banner; at the other the agent reached confirmation. Same software, different configuration. Which is oddly encouraging, because configuration can be fixed.

An AI agent readiness checklist for a booking business

What the method rewards, in the order an assistant meets it. This is what agent-ready booking means in practice.

1. A booking entry point an agent can find from your own website, not only from a marketplace.
2. Date, time and party size as real form fields, not pictures or a chat.
3. No captcha or device check on the part that only reads hours and free tables.
4. A consent banner that does not cover the booking form.
5. Typed answers to three questions: what you offer, when you are free, whether you can take this booking now (OpenAPI, MCP or another schema).
6. An llms.txt if you like, knowing it describes you and does not make you bookable.

Questions