« First in AI »: the spot holds, but there is one per language and per engine
Five runs of the same question: the brand named first stays the same 81 % of the time. The problem is not stability - it is that there is no single first place.
81%
of combinations keep the same brand first across five identical runs
In this article
Short answer: first place is fairly stable, and that is exactly what makes « we are first in AI » misleading. Ask the same question five times over: the brand named first does not change in 81 % of cases. But change the language or the engine, and it is no longer the same brand.
1 place
the median movement of a brand between two runs - one in two does not move at all
100
runs, zero failures: 4 sectors x 2 languages x 2 engines, five times each
In web hosting, Hostinger comes first in English, French and Spanish - and IONOS in German, on both engines. In payments it is Stripe almost everywhere, but Redsys in Spanish on Gemini. In hotel booking, Perplexity puts Booking.com first and Gemini puts Google Travel, on the same day, in the same language.
There is no first place. There is one per language and per engine, and nothing guarantees they resemble each other.
What we measured
The same reference question every measurement of ours has used since 23 August, asked five times over per combination. Four global sectors, two languages, two assistants that search. 100 runs, zero failures. Nothing changes between two runs: not the sector, not the language, not the engine, not the day. Only the assistant's own randomness speaks.
Cleaning comes before ranking, and that is not a detail. Our extractor still lets through fragments that are not company names. If they land in first position they manufacture an instability that is ours, not the engine's. So we keep only names that recur across several answers, and rank afterwards.
First place holds
| Measure | Raw | After label correction |
|---|---|---|
| Combinations keeping the same brand first | 12/16 (75 %) | 13/16 (81 %) |
| Brands that do not move a single place | 50 % | 50 % |
| Median movement of a brand | 1 place | 1 place |
| Largest movement observed | 5 places | 5 places |
One brand in two sits at exactly the same rank every time the question is asked again. The median movement is one place. Contrary to what is often said, the ordering is not a lottery.
We first got 38 %, and that figure was wrong: fragments such as « small teams » or « performance » were occupying first place. Cleaning before ranking gives 75 %, then 81 % once « hubspot crm » and « hubspot » are merged, since they name the same company. The instability we were about to publish was our own.
But there are eight first places, not one
| Sector | Perplexity | Gemini |
|---|---|---|
| Payments (en and fr) | Stripe | Stripe |
| Hotel booking (en) | Booking.com | Google Travel |
| Hotel booking (fr) | Booking.com | Booking.com |
| Web hosting (en and fr) | Hostinger | Hostinger, except in English |
| CRM (en) | HubSpot | HubSpot |
| CRM (fr) | no stable leader | no stable leader |
And the 24 August measurement, across more languages, extends the table. In German, the hosting provider named first is neither Hostinger nor a global player: it is IONOS, on both engines. Perplexity describes it in as many words as « in mehreren Vergleichen auf Platz 1 bzw. Testsieger ». In Spanish, Gemini surfaces Redsys in payments - the Spanish bank network - « muy utilizada en el mercado español ». Both companies are invisible in the other languages.
A German hosting provider is first in Germany. It does not exist in English. Both statements are true at once.
The French CRM case
It is the only combination in our measurement where no brand holds first place. On Gemini, five runs of the same question give three different leaders: Freshsales, HubSpot, monday. On Perplexity, Pipedrive alternates with a label that is not even a company.
There is no incumbent to dislodge there, and that is commercial information: in that market, first place is available. Elsewhere - Stripe in payments, Hostinger in English-language hosting - it is held by the same brand at every run.
What to do with this
A sentence like « we are first in ChatGPT » is not false: it is incomplete.
First in which language, on which engine, on which phrasing? Without those three, it cannot be checked.
Do not carry a first place from one market to another.
Leading in English says nothing about German, where a national player can hold the spot - and hold it on both engines at once.
Look at whether the top is held before attacking it.
When the same brand returns at every run, first place is a wall. When it changes every time, as in French CRM, it is open.
A single run cannot establish a rank.
Half of brands do not move, but the other half do - by up to five places in our measurement. A position is established over several repeats, never from a screenshot.
The limits
Four sectors, two languages, two engines for the repeated part.
Running shoes are excluded: the answers there cite numbered models our extractor discards, which produced a result that was not one.
The table extended to German and Spanish comes from the 24 August collection
, with a single run per combination. It shows that the leader differs by language; it does not establish its stability in those languages, for want of repeats.
Our extractor remains imperfect
, and one fragment still sits at the top of one combination even after correction. We publish both computations rather than one, and we name what remains.
« First » here means named first in the text of the answer.
It is not a ranking the engine publishes: it is the order in which it enumerates. We do not credit it with an intent to rank.
We promise nobody a place in ChatGPT's answers - nobody can do that honestly. What can be measured is where you stand: per language, per engine, per question asked. Including when the answer is « the spot is taken, and by someone other than you expect ».