JetBrains tests 12 AI models on PHP and Python
First seen Nov 15 Updated Nov 15, 23:41 UTC Heat 20 high evidence
JetBrains asked 12 recent AI models for a controversial programming-language opinion, 50 times per prompt and two prompts per model, 1,200 requests in all. Each request used a fresh conversation with no system prompt. The goal was to see whether models have distinct takes or repeat the same one.
- Who should care
- Developers curious about how consistent LLM answers are on opinion prompts.
Key facts
- 12 models from 10 labs in three countries
- 1,200 requests, all returned
- Two prompts, 50 runs each per model
- Collecting the responses cost USD 7.68
- Models include Claude Opus 5.5, Gemini 4 Argon, Grok 4.7, DeepSeek V4 Pro, Mistral Large 4
- Each prompt asked for 50 to 90 words
Timeline
Official
-
AI models hate Python and love PHP: we asked 12 of them 1,200 timesblog.jetbrains.com · Jetbrains
Evidence
High evidence: the claim comes from an official source (the company or project itself), or at least two independent sites reported it.
Details written Nov 15, 23:39 UTC from the sources listed above.