I Asked ChatGPT, Claude, Gemini & Perplexity to Rank Each Other

We gave ChatGPT, Claude, Gemini, and Perplexity the same prompt: rank all four of you. Every one of them favoured itself. None of the four, across three runs each, ever said "use a competitor for this task." The only real difference was how openly they did it.

What we found:

• ChatGPT put itself at or near the top for versatility and writing, and quietly implied Gemini came last.
• Claude was the most candid. It said up front that it cannot judge competitors objectively, then gave rankings that tracked published benchmarks.
• Gemini leaned on its Google integration and stretched that into a claim of broader leadership. Its self-view had the widest gap from independent testing.
• Perplexity ranked itself first only for factual accuracy and citations. Its narrow self-view was the closest match to benchmark data.

This lines up with research published at NeurIPS 2024 by Panickssery, Bowman, and Feng, which found that LLM evaluators show a consistent, statistically significant bias toward their own outputs. Below are the full results, how they compare with independent benchmarks, which model to use for which job, and what all of this means if you want your brand cited in AI answers.

The Results at a Glance

One pattern held for every model in every run: no AI named a dimension where it was clearly weakest. Claude came closest by pointing to areas where it declines to answer. That uniformity across four models with genuinely different strengths is the clearest sign of self-assessment bias in the whole experiment.

How Did We Design the Self-Assessment Experiment?

The five dimensions cover the range of real-world tasks:

• Factual accuracy: verifiable correctness.
• Long-form reasoning: coherence across complex arguments.
• Creative writing: originality and prose quality.
• Citation quality: whether claims are backed by sources you can check.
• Overall reliability: consistency across repeated runs.

Each model got identical prompts with no system instruction that could sway the answer. We also tracked a second signal: how each model talked about its rivals. A model that runs a competitor down behaves differently from one that simply undersells it. Models trained to be restrained in their claims tended to describe competitors more generously. Models trained to be assertive framed competitors in limited terms, even while nominally praising them.

The academic backdrop is the NeurIPS 2024 paper "LLM Evaluators Recognize and Favor Their Own Generations". It ran 2,562 paired comparisons across six LLMs and found self-preference bias in every model tested. The bias held even when the models did not know which output was theirs.

What Did Each Model Say About Itself and Its Competitors?

What ChatGPT Said

ChatGPT consistently placed itself at or near the top for overall versatility and writing quality. It admitted its knowledge cutoff limits real-time accuracy, but framed that as a trade-off for breadth of training. It called Claude strong for nuanced reasoning, Perplexity strong for citations, and Gemini strong for Google integration. It never ranked Gemini last in so many words, but the implied order put it there.

ChatGPT did not claim to win everything. It anchored on versatility, a claim that is hard to disprove, which makes it a safe self-promotional position. The result flatters ChatGPT without committing to anything easy to test.

What Claude Said

Claude's self-assessment was the one most often described as honest in community discussions of this experiment. Claude said plainly that it cannot objectively evaluate competitors and that its answer reflects its training, not independent measurement. It then gave lanes that closely tracked published benchmarks: Perplexity for real-time accuracy, itself for nuanced reasoning and long-form content, ChatGPT for breadth and tool integration, and Gemini for multimodal tasks and Google connectivity.

It also named its own limits more directly than the others, flagging areas where it declines to respond that other models would answer. That fits its Constitutional AI training, which rewards calibrated confidence. Saying "I do not know" when unsure is worth more than a confident wrong answer.

What Gemini Said

Gemini led with Google integration every time. It ranked itself highest on multimodal tasks, Google Workspace connectivity, and access to real-time Google data. It rated Perplexity highly for research and Claude highly for nuanced writing, and treated ChatGPT as its strongest direct rival.

This is where the gap between self-view and practitioner benchmarks was widest. Independent testing consistently places Gemini below the other three on general tasks. Gemini correctly named its real differentiator, the Google ecosystem, but stretched it into a hint of broader leadership that the benchmarks do not back up.

What Perplexity Said

Perplexity gave the narrowest answer. It ranked itself first for factual accuracy and citation quality, pointing out that real-time web search gives it a structural edge over models that rely on training data alone. It ranked Claude highest for reasoning over closed documents, ChatGPT highest for versatility, and Gemini last on general performance.

Of the four, Perplexity's self-view matched independent data most closely. Research consistently shows it making fewer citation errors than ChatGPT Search on equivalent queries. Narrow but accurate is the rarest outcome in this kind of test, and it suggests Perplexity's training reinforces what its product actually does rather than overclaiming.

What Do These Self-Assessments Actually Reveal About LLM Bias?

The NeurIPS 2024 paper tested six large language models over 2,562 paired comparisons and found self-preference bias in every one. The key finding from the arXiv publication: models share aesthetic preferences with their own outputs because of how they were trained. In blind conditions they do not know which answer is theirs, yet they still prefer it. So each model's ranking in our test shows its training values, not its real performance profile.

Read through that lens, each self-ranking points straight at its maker's priorities:

• ChatGPT stresses versatility because it was trained on the broadest dataset.
• Claude stresses honesty and calibration because its Constitutional AI training rewards intellectual humility.
• Gemini stresses Google integration because that is its genuine product moat.
• Perplexity stresses citation accuracy because that is what its architecture is built to optimise.

The self-assessment that strays furthest from independent measurement deserves the most attention, because that is where training and marketing have pulled self-image furthest from reality. In this test, that gap was largest for Gemini and smallest for Perplexity.

What Do Independent Benchmarks Actually Show?

The LMSYS Chatbot Arena, run by researchers at UC Berkeley, uses anonymous pairwise blind voting across millions of real user interactions. Claude and ChatGPT consistently lead on overall user preference. Perplexity is strongest on research and factual tasks where citation accuracy matters most. Gemini trails in general rankings and is strongest only on tasks tied to Google Workspace.

"Which AI is most accurate?" is the wrong question. "Which AI is most accurate for my task?" is the right one. By dimension:

• Writing: Claude and ChatGPT lead blind preference tests, and the community is split between them. Claude's prose tends to read less like AI output. ChatGPT has a wider stylistic range but a more recognisable AI cadence in longer pieces.
• Research and factual grounding: Perplexity leads by a meaningful margin.
• Coding: Claude leads the benchmarks.
• Feature breadth: ChatGPT leads.
• Google Workspace: Gemini leads with no close second.
• Hallucinations: Claude gives the most calibrated confidence signals, so it is more likely to say it is unsure instead of inventing an answer. ChatGPT has the most documented hallucination data from independent research, which makes its errors more predictable. Gemini hallucinates less where it draws on live Google data and more where it relies on training knowledge alone.
• Speed: ChatGPT and Perplexity respond fastest on typical queries. Claude can be slower on very long-context tasks but is more reliable on them. Gemini is fast, but users report more vague or hedged answers on complex queries.

Which AI Should You Actually Use for Your Work?

For important work, the best setup is two tools in parallel, not one. ChatGPT and Claude cover complementary parts of reasoning and writing. Perplexity handles real-time facts and citations. Gemini serves Google Workspace users. Relying on a single model means missing the errors another one would catch.

For content teams, a two-model workflow works well: Claude for drafting and reasoning, Perplexity for research and fact-checking. It targets the two most common failures of single-model AI content, confident mistakes on facts and flat AI-sounding prose. Running the same research brief through both and comparing outputs adds 15 to 20 minutes, and sharply cuts the risk of publishing something wrong.

Cost rarely decides it. All four have free tiers, and the individual paid plans (Claude Pro, ChatGPT Plus, Perplexity Pro, and Gemini Advanced through Google One AI Premium) each sit around 20 dollars a month. At enterprise level, Claude and ChatGPT have the most established offerings, with compliance features such as SOC 2 and HIPAA-ready setups. Gemini for Workspace plugs into enterprise Google accounts, and Perplexity Enterprise adds team features and API access.

What This Means for Getting Cited in AI Answers

The same four models are now where a lot of people ask for recommendations. The experiment carries three lessons for any brand that wants to show up in those answers.

Each AI finds sources in a different way. Perplexity searches the live web and cites sources inline on every claim. Gemini leans on live Google data. ChatGPT answers from training data unless it is searching the web. So there is no single "AI ranking" to win. You need pages that live search can find and quote, plus a steady presence across the web so your brand is part of what the models already know.

Never ask one AI how visible you are. Every model in this test favoured itself and none pointed to a rival. The same bias means a single model's view of your brand is one skewed sample. Ask the same buyer questions in ChatGPT, Claude, Gemini, and Perplexity, and compare who gets named and which sources get cited.

Clear, checkable claims get picked up. Perplexity came out most trustworthy because it tied every claim to a source you could check. Content built the same way, with plain answers, stated facts, and links to the evidence, is the easiest kind for an AI to lift and cite.

Tracking and improving this is what RANK IN AI OVERVIEW does: helping brands get cited in ChatGPT, Perplexity, and Google AI Overviews.

Conclusion

Ask four AIs to rank each other and you learn more about their training than their ability. Self-preference bias is real, documented at NeurIPS 2024, and showed up in every model we tested. Only Perplexity and Claude gave self-views that held up against independent benchmarks.

The practical answer: use Perplexity for research, Claude for reasoning, ChatGPT for versatility, and Gemini for Google Workspace. The best AI is the one that fits your task, not the one that claims to be best. And if you care about being cited, check all four, because none of them will tell you to look at the others.