The same business, the same question: four AI models give different results
We put the same questions to four AI models, for two businesses, in the same period. Two models named the business in more than half the answers. Two named it in fewer than a third.
Mentions by model
- Claude9 of 1656.3%
- Gemini9 of 1656.3%
- ChatGPT5 of 1631.3%
- Perplexity5 of 1631.3%
What we assessed
Two businesses in Albania, in the same period. For each we built the questions a customer actually asks, and most of them do not mention the business by name, because otherwise the answer would come from the question itself.
The same 16 questions went to four AI models: Claude, Gemini, ChatGPT and Perplexity. That is 64 stored answers, each with its date, its model and its language.
The figures below are aggregated across both businesses. Neither business is named.
Two models saw it. Two did not.
Claude named the business in 9 of 16 answers, and Gemini did the same in 9 of 16.
ChatGPT named it in 5 of 16, and Perplexity also in 5 of 16.
Between the model that sees the business most often and the one that sees it least there is close to a factor of two. The same question, the same business, the same period.
In half the questions the models did not give the same answer
The same 16 questions went to all four models. In 8 of them the models did not agree: some named the business and some did not.
Half the questions. This is the part a business cannot see on its own, because anyone asking a single model gets a single answer and has nothing to compare it against.
Where the models disagreed
8 of 16
Why it happens, and what we cannot say
Models do not return a ranked list of pages the way a search engine does. They build an answer out of what they have read and what reaches them in the moment, and each weighs that material differently.
What we cannot say from this data: why this particular model sees the business and that one does not. Answering that needs the sources behind each answer, read one by one. We store every answer precisely so that work is possible, but it is not the question answered here.
What it means for a business
If you check on one model only, you have seen a quarter of the picture, and depending on the model you chose you may come out noticeably better or worse than you really are.
The same holds for competitors. A company that comes first on one model may not appear at all on another, and both are true on the same day.
The limit of this finding
Two businesses and 64 answers are a small base. This is what we assessed, not a law of the market, and we put the number first precisely so it is judged that way.
Each question was asked once per model in this period. Repetitions are part of the approved method and did not enter this finding.