Hallucinations have improved, but the average for all models is still around 8-9%. Likely the study I read before was from last year, so the higher hallucinations rate is from that.
Even the best model hallucinates an average of around 1 in 100 times and OpenAI has stated publicly that hallucinations are mathematically impossible to eliminate entirely.
I don’t trust it wholesale. No one who knows what they’re talking about trusts it wholesale. Hallucination rates vary depending on who you ask. And you’re wrong about vibe coding, it works great if you’re working on some random side project and not working with a team that has to push to production
Generative AI has an average error rate of 9-13%. Nobody should trust it wholesale and what it spits out.
It has some excellent use cases. Vibe code/sysadmin/netadmin’ing are not one of those things.
Where does this 9-13% number come from?
There was a study done several months ago. I’ll try to find the source again and link it here in a comment edit.
[EDIT]
Couldn’t find the study, but here is an infographic that has similar data: https://infogram.com/ai-hellucination-report-2025-1h9j6q7mz7q354g
Hallucinations have improved, but the average for all models is still around 8-9%. Likely the study I read before was from last year, so the higher hallucinations rate is from that.
Even the best model hallucinates an average of around 1 in 100 times and OpenAI has stated publicly that hallucinations are mathematically impossible to eliminate entirely.
I don’t trust it wholesale. No one who knows what they’re talking about trusts it wholesale. Hallucination rates vary depending on who you ask. And you’re wrong about vibe coding, it works great if you’re working on some random side project and not working with a team that has to push to production