Google AI Overview gets Pokémon facts wrong, gamers are infuriated – and it points to a dangerous problem

'Where is the loaded dice in Pokémon Sword and Shield?' a gamer asked Google. Pretty simple and entirely innocent, you'd think. But Google was unable to answer the question correctly, highlighting a much wider issue in the media landscape.

Google's AI overview is taking over search, much to the dismay of publishers (like us). But as this recent Pokémon example shows, you can't always trust the information in those summaries as the AI just isn't up to scratch yet. The result in this case is infuriating to gamers, as shown by a heated discussion on Reddit, but misleading answers can be more serious if users take the answer given as gospel - such as with health advice or topics related to purchasing decisions.

Here, the AI informs users how to get the loaded dice in Pokémon Sword and Shield confidently (if AI could have that emotion) stating that, "you go to the back entrance of the Celadon mansion, past that then behind the Celadon Department Store." However, as the indignant players assert on the forum, that's not right at all, because everyone knows you can't get loaded dice in this version of the game. Whoops!

Google knows nothing about Pokemon. This isn't the first time they got something wrong. from r/PokemonSwordAndShield

Disinformation is one of the major issues facing the world today, with serious social and political implications. Google's laissez-faire attitude to presenting AI summaries of topics that are wrong is negligent at best and dangerous at worst. To roll out this technology while it is unable to provide correct answers will fuel the spread of false facts – and Google has intentionally made it so appealing to bypass the source with these handily digestible chunks of information.

You may think I'm biased, as someone who literally works at the source and is being negatively influenced by the AI overview feature, but one thing I know is that human journalists and content creators (from reputable sources) fact check and will, on the whole, present the truth to the best of their ability. And going directly to websites gives you much needed context about what you are reading in terms of biases and reputability. The AI Overview encourages people to ignore the sources (sure, they're listed there but is anyone really looking? We just want to be served up those sweet chunks of easily readable and, crucially, quick information).

It's something known by most, as the Reddit thread reflects. One user makes a very good point about expertise in relation to being able to spot wrong information.

"Something I've learned is that google’s AI rarely knows what it’s talking about. I used to think it was decent when I asked it questions about topics I knew nothing about, the moment i asked it a Pokémon question I realized everything it’s told me before has probably been a lie".

Others talk about being able to turn the feature off (a good point but only relevant if users know they aren't getting quality answers through the overview – and that isn't necessarily the case). There is no-one defending the feature, and tons of comments talking about how Google search has gone downhill – with this as a headline example.

Of course, AI gets smarter all the time, and Google is probably relying on it getting better whilst in action. But in the meantime, false information is out there via the platform most used for information-gathering. And it goes much deeper than loaded dice in Pokémon.

See here for some AI productivity tools that are actually useful.