Claude (Anthropic), ChatGPT (OpenAI) and Gemini (Google) keep evolving and overtake each other depending on the use case: a ranking published six months ago says little about your project today. The right choice is made on your own cases, with a few concrete criteria. Here is the method I apply before integrating a model into a website or an agent.
Three families, several sizes
Each provider offers several models: large models, more capable but more expensive and slower, and smaller models that are fast and cheap. The choice of size often matters as much as the choice of provider:
- a small model is enough to classify messages, extract information from a form or answer an FAQ;
- a large model is justified for writing long texts, reasoning over several steps or driving an agent with many tools.
Test on your real cases
The most reliable method is still to test the models on real examples from your business:
- gather 20 to 30 typical cases: real customer questions, documents you handle, answers you expect;
- run them through two or three models, with the same instructions;
- score each answer on a simple grid: accuracy, tone, respect of the requested format;
- note the cost and response time at the same time.
Half a day of tests like this tells you more than any general ranking.
The criteria that matter depending on the use
- Writing quality and tone: the gap shows mostly on long or nuanced texts.
- Reasoning: for multi-step tasks, the models do not all behave the same way.
- Tool use: for an agent, how reliably the model picks the right tool and fills in its parameters is decisive.
- Document length: how much text the model can take in at once (its context window) varies, as does its ability to use all of it well.
- Language: quality in French, and even more so in Arabic and Tunisian dialect, is not identical from one model to another.
- Response time: important for a live assistant on a website or WhatsApp.
- Data hosting: the models are available directly from their provider, but also through the major clouds (Claude on AWS and Google Cloud, OpenAI models on Azure, Gemini on Google Cloud), which can matter when choosing the region your data goes through.
Estimate the cost before choosing
APIs are billed on the volume of text sent and received, counted in tokens (a token is roughly a syllable or a short word). The order-of-magnitude calculation is simple:
monthly cost ≈ requests per month
× (input tokens × input price + output tokens × output price)For an agent, count several exchanges with the model per request, since every tool call adds one. A price gap that looks small per request becomes significant at several thousand conversations a month: that is where small models make the difference.
An architecture that does not lock you into one choice
The field moves fast: you need to be able to switch models without rebuilding everything. To do that, business logic should never call a provider’s API directly, but go through a common interface. For example, in PHP:
interface AiModel
{
public function answer(string $instructions, array $messages): string;
}
final class ClaudeModel implements AiModel { /* call to the Claude API */ }
final class GeminiModel implements AiModel { /* call to the Gemini API */ }Switching models then comes down to changing one line of configuration, and you can even use a small model for simple tasks and a large one for the most complex.
In practice
- Define the task precisely and gather real cases.
- Test two or three models, including at least one small one.
- Compare accuracy, tone, cost and response time.
- Pick the smallest model that gives satisfactory results.
- Keep the architecture open, and run the test again in a few months.
Going further: my expertise: AI agent developer · read also: Integrating AI into your site without putting your data at risk
