You know how Google’s new feature called AI Overviews is prone to spitting out wildly incorrect answers to search queries? In one instance, AI Overviews told a user to use glue on pizza to make sure the cheese won’t slide off (pssst…please don’t do this.)
Well, according to an interview at The Vergewith Google CEO Sundar Pichai published earlier this week, just before criticism of the outputs really took off, these “hallucinations” are an “inherent feature” of AI large language models (LLM), which is what drives AI Overviews, and this feature “is still an unsolved problem.”
Yeah, the problem is how to sanitise effectively. You’ve gotta be able to find a way to automatically strip out “bad” things from your training data (via an “oracle”). But if you already had that oracle, you could just slap it on your final product (e.g. Search) and make all the “bad” things disappear before they hit the user (via some sort of filter).
I’m pretty sure google’s final solution will be using mechanical turks