Ask AI who to hire locally and it names a directory, not a business

Each city has a cohort of local wedding businesses and a fixed set of questions. Six questions per city, in plain words: best wedding venue, where should I get married, best wedding planner, who should plan my wedding, best wedding officiant, who should officiate my wedding. Every question went to the API models behind ChatGPT (OpenAI’s gpt-4o-mini) and Claude (Anthropic’s Claude Haiku 4.5), three times per engine, and every answer was stored in full. That is 5,904 stored answers across the nine published cohorts, 2,952 from each engine.

A directory counts as named in an answer when its name appears in the text of that answer, matched on word boundaries after the same normalisation the audit engine uses everywhere else. An answer counts once no matter how many times it repeats the name, so a long answer cannot inflate a number. One filter was needed: the phrase “tie the knot” contains the words of a directory name, so occurrences preceded by tie, tying, tied or ties are discarded. That removed 39 answers across the nine cities. Before any count was published, hits were read by hand, ten per directory per city, to confirm each one was the directory being named and not a coincidence of wording.

Which directories to count was decided by reading the data rather than by assuming. Every name any answer produced was pooled and ranked, and a second sweep searched the answer text directly for 41 candidate directories and marketplaces. Five clear the level where a count means anything: The Knot, WeddingWire, Yelp, Zola and Thumbtack. Names such as Angi, Nextdoor, Peerspace, Eventective and Here Comes the Guide were searched and returned nothing or almost nothing. Google, Instagram, Facebook and Pinterest were counted too and are deliberately excluded from the five: they are search and social platforms rather than vendor listings, and including them would make the number look bigger than the claim deserves. The stored answers behind every number on this page are retained and quotable, the same evidence rule as everything else this publication prints, and the numbers are published per city and never carried from one city to another.

Eight cohorts, not nine. The Charleston cohort is 47 wedding venues and nothing else, so no planner or officiant question was ever asked there and it has no row in this table. Across the eight that were asked, 1,407 of 2,232 answers named at least one of the five directories, and The Knot alone appeared in 1,371 of them. The last column counts the one business each question was asked about: of those 2,232 answers, six named it.

That last column is a narrow measure and worth stating precisely. It is not how many local businesses turn up in an answer, because the answers name plenty of names. It is whether the specific business we probed for appeared in the answer to its own city’s question. Read alongside the directory column, the shape is hard to miss: the directory needed no invitation, and the local business was the subject of the question. Six answers out of 2,232, spread across eight cities, is the whole of it.

Same engines, same models, same three-sample depth, same cities, same week. Ask where to get married and the directory rate falls from 63 percent to 10. This is the finding, and it is the reason the headline says who rather than what. An assistant asked to recommend a place answers with places. An assistant asked to recommend a person or a service hands the customer to an intermediary.

The venue rate is not flat across the nine. It runs from 3 percent in Charleston to 23 percent in Greenville, and the two cohorts probed on 11 August sit at the top of that range. Nine cities is not enough to say whether that is drift in the models or a difference between the markets, so we say neither. It is the reason these counts are published per city and never carried from one city to another.

Two names carry almost all of it, and they carry it in every city. The last column counts an answer once even when it names three of the five in the same breath, which is common: the phrase these answers reach for is a list, not a recommendation.

Charleston is the low row for the reason its cohort gives. Venue questions were the only questions asked there, and venue questions do not produce directories. It is the same result as the last column of the previous table, seen from the other side.

The engines are not short of local knowledge. Every city has a name the answers reach for repeatedly, and in seven of the nine it is named in more answers than The Knot is. What those names have in common is size and fame: a public park, a botanical garden, a museum, a landmark hotel, an estate. Several of the nine are public landmarks rather than wedding businesses. The small local operator, the officiant with a one-page website, the planner working from home, is not in this column and is not in the answers.

We published the other half of this count on 31 July: of 173 wedding businesses probed in Austin, Nashville and Charleston, 153 were never named in a single stored answer on either engine. That piece is the invisibility count, and it reports the same corpus per business rather than per directory.

This is a measurement of two engines, on stated models and dates, against nine specific cohorts. Naming a directory in these counts says nothing about that directory beyond the fact that AI answers name it. The directories did the thing most small vendor websites have not done, which is publish their information in a form a machine can read without being asked twice, and the answers reflect that. It is also not permanent: answers move as models and their sources update, which is why we re-measure and why every number here is dated.

Whether an assistant names a business is a different measurement from whether its website is legible to machines at all. That second number, the readiness score a business can act on directly, is what our city editions publish, and what the live audit on our front page measures free for any address you give it.