Why this is hard to judge
Traditional SEO reporting is mature. Rankings, sessions, conversions, all standardized. Nothing equivalent exists yet for AI answers, so a monthly report can look complete while containing no information about whether an assistant names you. Most agencies are not being dishonest about this. They genuinely added AI language to their pitch before they added AI measurement to their process.
That is fine, as long as you know which one you are buying. The questions below sort it out in a single call, and a good agency will enjoy being asked them.
The four non-answers
- "We optimize for AI as part of our SEO work." This is true of everyone and commits to nothing. Ask what is different about it.
- "We add AI-optimized schema." Structured data is a public standard and is not an inclusion mechanism. Nobody's version of it is AI-optimized.
- "We have seen great results across our client base." Ask about your account, on your questions, with dates.
- "AI search is still too early to measure." Anyone can measure it by hand in two minutes. This answer means it is not being done.
What a good answer sounds like
Specific and slightly boring. Something like: we track these forty questions for you, we check them monthly across two assistants, you were named in eleven of them in July and fourteen in August, the three we are losing are all same-day questions, and here are the two pages we are writing about it. That answer contains numbers, dates, question wording, and a next action.
If the answer is no
That is not automatically a reason to leave. The right response is to add measurement, either by asking them to start reporting it or by running the check yourself, and then to judge the existing work by whether it moves. Plenty of good ordinary SEO work improves AI visibility as a side effect. You just have no way of knowing which parts until somebody measures.
Eight questions for your next agency call
Ask them in this order. The first four separate measured work from talk. The last four tell you whether the work itself is sound.
| Ask this | A real answer looks like | A non-answer looks like |
|---|---|---|
| Which specific questions do you track for us? | A list you can read, in customer wording, fixed since a stated date. | "All the relevant ones for your industry." |
| How often do you check them, and in which assistants? | A cadence and named assistants, with the last run date. | "Continuously", with nothing to show. |
| In how many of them were we named last month? | A number, and the number for the month before. | A description of activity instead of a result. |
| Who is being named instead of us? | Named competitors, per question. | "The usual big players." |
| Which third-party pages are you getting us onto? | Named pages, with why each one is retrieved for our trade. | "We are building citations across the web." |
| What did you change on our Business Profile, and when? | Category, services, description, with dates. | "We keep it optimized." |
| How are you generating review text that names the job? | A described process the client's staff actually follow. | A review-gating or incentive scheme, which is a policy violation. |
| What would you show us if it did not work? | A specific diagnostic and a change of approach. | Reassurance. |
Print this and take it to the call. The value is in asking all eight in one sitting: any single question can be talked around, the set cannot.
You are looking for the term
AI visibility monitoring
The thing you are asking your agency to prove exists is called AI visibility monitoring: a fixed question set, run on a schedule, recording mention rate, citations, and competitor share of voice. Whoever does it, in-house or agency or software, it is the same four numbers.
- Should I hire a separate AI visibility agency?
- Usually not. The work overlaps heavily with local SEO, and splitting it across two vendors tends to produce two partial pictures. Add measurement first, then decide.
- Can I just do the measurement myself?
- Yes, by hand, for a small question set. It takes a couple of minutes a month and it is enough to hold any vendor accountable.
- 01Top ways to ensure your content performs well in Google's AI experiencesGoogle Search Central
- 02Guidelines for representing your business on GoogleGoogle Business Profile Help
- 03Tips to get more reviewsGoogle Business Profile Help
- 04GEO: Generative Engine OptimizationAggarwal et al., arXiv
Every link above was opened and confirmed reachable on September 1, 2026.
The automated version
See what ChatGPT, Claude, Gemini and Perplexity say about your business
The free scan runs the buyer questions for your trade and city across all four assistants, shows who gets named instead of you, and lists the sources those answers came from. No signup, about thirty seconds.
Run my free scan