Google Admits It Faked Gemini AI Demo

Google made waves this past week when it revealed its new Gemini multimodal AI. The firm reportedly hopes that Gemini will help it combat OpenAI and its GPT-4 model. The demo video appears to show the machine effortlessly recognizing objects and carrying on a conversation in real time. However, a new report reveals Google staged this video to make Gemini look more impressive.

Gemini is Google's largest and most capable AI model. For the 1.0 release, the company created three versions of Gemini: Ultra, Pro, and Nano. Ultra is the most complex, designed to run on servers. Gemini Pro is the middle-of-the-road version intended to run a range of tasks in the cloud. Gemini Nano was designed to be more efficient for on-device usage. Gemini Nano is already powering some features on the Pixel 8 and Pixel 8 Pro. Bard, Google's generative AI chatbot, is now running on Gemini Pro.

Along with the announcement, Google posted a demo video showing disembodied hands drawing and moving objects around. An artificial voice responds proactively to what happens, effortlessly answers questions, and fulfills requests. It would be impressive if it happened like that.

According to Bloomberg, the demo is more than a little misleading. For starters, the output is much slower. There's a disclaimer in the video that responses have been sped up, but that's not the most egregious deception. Gemini wasn't even watching the video—all the replies we hear result from showing the AI still frames from the video and feeding it text prompts.

Google stresses that all the outputs in the video are real. Google's Oriol Vinyals, who co-leads the Gemini effort, notes on Twitter that the demo video was intended to "inspire developers." However, the accuracy of Gemini's replies should have been inspiring enough. Making the model seem so perfectly interactive is misleading, and that's not great when the public and regulators are already distrustful of this new breed of AI.

This isn't even the first time Google made an unforced error when trying to show off its AI tools. In the wake of Microsoft's ChatGPT announcement early in 2023, Google assembled a hastily organized demo of its Bard AI. The model made several prominent mistakes, which soured Google's big AI debut. Despite being a long-time leader in machine learning research, including work on the transformer algorithms that underlie generative AI, Google just can't figure out how to hype machine learning.