Every few months the benchmark rankings shift. A new model drops, someone posts a chart showing it beats GPT-4 on MMLU, and suddenly everyone's reconsidering their stack. This is mostly noise.
The dimensions that matter in production are different from the dimensions that academic benchmarks measure. Instruction following consistency under edge cases. Reliability over thousands of API calls. Behaviour when the context window is nearly full. How the model handles ambiguous inputs rather than clean test prompts. Cost at the volume that makes your unit economics work.
Kami dah build dengan ketiga-tiga models ni dalam production systems. Ini adalah what we actually found.
What it is
Anthropic's Claude family currently spans Haiku (fast, cheap), Sonnet (balanced), and Opus (most capable). The Claude 4 generation represents a significant step in instruction following and reasoning. For production agentic systems, Claude Sonnet 4 is the working default — Opus for tasks requiring deepest reasoning, Haiku for high-volume classification or extraction where cost matters.
What it does well
- Instruction following is consistently the best of the three — complex, multi-part prompts with many constraints
- Longer outputs stay coherent and on-task; less drift than GPT-4 on extended generation
- Tool use (function calling) implementation is clean and reliable for agentic workflows
- 200K context window (Sonnet) handles large document sets without degradation near limits
- More predictable refusal behaviour — easier to reason about what it will and won't do
- Bahasa Malaysia comprehension is solid for a non-English-first model — better than GPT-4 for code-switched Manglish
- Native image generation not available — vision input only
- Smaller third-party ecosystem than OpenAI; some frameworks assume GPT as default
- API pricing at Opus tier is premium — Sonnet is competitive but Haiku vs GPT-3.5-turbo depends on use case
- No Azure hosting option — data residency in US by default (relevant for strict compliance)
Why instruction following matters for agentic systems: When an AI agent is running a multi-step workflow, small deviations from instructions compound. If the model takes a slightly wrong interpretation at step 3, step 7 is built on that error. Claude's consistency on complex, constrained instructions is why we use it for IRIS — the difference between a model that follows 19 out of 20 constraints and one that follows all 20 is the difference between a system that needs constant supervision and one that doesn't.
What it is
OpenAI's GPT-4o (omni) is the current production flagship — combining text, vision, and audio in a single model. GPT-4o-mini is the lower-cost alternative for high-volume tasks. The OpenAI ecosystem is the most mature of the three: the widest third-party tool support, the most Stack Overflow answers, and the framework most tools default to when they say "LLM."
What it does well
- Multimodal capability is genuine — vision + text in single call, useful for document OCR + analysis pipelines
- Largest developer ecosystem; most libraries, templates, and examples are written for OpenAI first
- Azure OpenAI deployment available — data residency options including Southeast Asia regions
- Function calling is well-documented with extensive production examples available
- GPT-4o-mini has good cost/performance ratio for classification and extraction tasks
- Structured output mode (JSON mode) is reliable for extraction pipelines
- Instruction following on complex, multi-constraint prompts less consistent than Claude in our experience
- Context window 128K (GPT-4o) — sufficient for most but smaller than Gemini and Claude's limits
- More sycophantic — tends to agree with incorrect user premises rather than correct them
- Malay language support is functional but code-switched Manglish trips it up more than Claude
- API reliability has had more incidents historically than the other two
What it is
Google's Gemini family runs from Flash (fast, cheap, capable) to Pro (production-grade) to Ultra (research-grade). The headline feature is context window size — Gemini 1.5 Pro supports up to 1 million tokens, which is in a different category from the others. Gemini 2.0 Flash in particular has become surprisingly competitive on cost/performance for routine tasks. Deployment via Vertex AI on Google Cloud gives enterprise data controls.
What it does well
- 1M token context window is genuinely differentiated — load entire codebases, entire legal case files, entire product catalogues
- Gemini 2.0 Flash is cost-competitive and fast — good for high-volume simple tasks
- Deep Google ecosystem integration: Google Drive, Docs, Sheets, Gmail via Workspace APIs
- Vertex AI deployment on Google Cloud with Singapore region available — data stays in SEA
- Multimodal support including video understanding — unique capability
- Malay language support is improving; Google Translate heritage gives it some advantage on formal BM
- Instruction following consistency lags behind Claude for complex agentic prompts in our testing
- Ecosystem maturity behind OpenAI; fewer production examples and community solutions
- API surface has changed more frequently — migration overhead between versions
- Performance in long context is good but not always consistent — "lost in the middle" problem exists
- Google's enterprise sales process is heavier than Anthropic or OpenAI for smaller teams
Head-to-head: the dimensions that actually matter
| Dimension | Claude Sonnet 4 | GPT-4o | Gemini 2.0 Pro |
|---|---|---|---|
| Instruction following (complex) | ★ Best | Good | Good |
| Context window | 200K tokens | 128K tokens | 1M tokens (1.5 Pro) |
| Agentic / tool use | ★ Best | Good | Improving |
| Multimodal (vision) | Input only | ★ Input + output | Input + video |
| Bahasa Malaysia / Manglish | ★ Best | Functional | Good on formal BM |
| Cost (mid-tier model) | Competitive | Competitive | ★ Cheapest (Flash) |
| Data residency (SEA) | US only | Azure SEA available | GCP Singapore available |
| Ecosystem maturity | Growing fast | ★ Most mature | Less mature |
| Our primary use: agentic AI | ★ Default choice | Secondary | Not used for agents |
So which one for your use case?
What TTD uses and why
Primary model for all production agent work: Claude Sonnet 4. The instruction following advantage is not marginal — it's structural for agentic systems. IRIS runs on Claude. Every production agent we've built for clients runs on Claude. This isn't brand preference; it's the outcome of actually running these systems under load and watching where they fail.
Secondary: GPT-4o for vision-heavy tasks. When a workflow involves image input — receipt processing, document layout analysis, form extraction from scanned documents — GPT-4o's native vision capability is cleaner to work with than the alternatives. We use it as a specialist rather than a general-purpose model.
Gemini Flash for cost-sensitive high-volume classification. When we need to run thousands of short classification calls and cost matters more than frontier instruction following — Gemini 2.0 Flash is the right tool. Google Cloud Singapore also gives us a data residency option for clients with SEA requirements.
We do not use a single model exclusively and we do not recommend that clients do either. The right architecture uses each model for the task it's best suited for, with abstraction layers that allow swapping if the landscape changes — which it will, probably within 12 months.
Pricing at production scale — the numbers that matter
Benchmark comparisons rarely include cost because it changes frequently and depends heavily on use case. But for Malaysian SME building production systems, cost per token is often a real constraint. Here is the current rough picture as of mid-2026 — always verify against official pricing pages before committing.
| Model | Input (per 1M tokens) | Output (per 1M tokens) | Best for cost efficiency |
|---|---|---|---|
| Claude Haiku 4 | ~$0.80 | ~$4.00 | High-volume extraction, classification |
| Claude Sonnet 4 | ~$3.00 | ~$15.00 | Production agents, complex reasoning |
| GPT-4o-mini | ~$0.15 | ~$0.60 | Cheapest capable model for simple tasks |
| GPT-4o | ~$2.50 | ~$10.00 | Multimodal, structured output |
| Gemini 2.0 Flash | ~$0.10 | ~$0.40 | Highest volume, cost-sensitive workloads |
| Gemini 1.5 Pro | ~$1.25 | ~$5.00 | Long-context document analysis |
Practical implication: for an agent that processes 10,000 documents a month with average 2,000 tokens input and 500 tokens output per document — that is 25 billion tokens a year. At Claude Sonnet pricing that is roughly USD 4,500/month. At Gemini Flash that is USD 375/month. The right model for your architecture might be a mix of both: Claude for the reasoning steps, Gemini Flash for the pre-processing extraction steps.
Bahasa Malaysia and Manglish — the dimension vendors ignore
For Malaysian applications, language performance in BM and Manglish is a real evaluation criterion that most Western comparison articles skip entirely. Our experience across the three models:
Claude handles code-switched Manglish — sentences that mix BM and English naturally at clause boundaries — better than the others. It doesn't try to force the output into pure English or pure BM. When we instruct it to write in the voice we use in The Lab — technical content with Manglish framing — it stays in register more consistently than GPT-4o or Gemini.
GPT-4o handles formal BM adequately but code-switched content gets normalised toward standard English more aggressively. It will answer a Manglish question in fluent English rather than responding in kind, which is often not the desired behaviour for Malaysian-facing chatbots.
Gemini has a corpus advantage from Google Translate and Search data that covers formal Bahasa Malaysia well. For generating formal BM government documents or regulatory text, it is competitive. For conversational Manglish, it lags behind Claude.
If your product is talking to Malaysian users in their natural register — not formal BM, not pure English — test this dimension specifically with your actual prompts. The gap is real and it matters for user experience.
One honest caveat about this comparison
The LLM landscape moves fast. Claude 4, GPT-4o, and Gemini 2.0 are current as of mid-2026 but specific capability rankings can shift with model updates. The structural observations — Claude's instruction following advantage for agents, Gemini's context window leadership, GPT's ecosystem maturity — are more durable than specific benchmark numbers.
Treat any comparison article, including this one, as a starting point for evaluation rather than final answer. The most important test is running your specific workload against each model with your actual prompts and measuring what matters to you. An hour of actual testing beats any benchmark chart.
Next in this pillar: MCP — Model Context Protocol, the emerging standard for giving AI agents structured access to external tools and data sources. If you're building anything that connects an LLM to external systems, this is the article to read next.