Is Pixie anything but the ‘new Google Assistant’ done right?

Pixie first came to light in December. The Information revealed the existence of an AI assistant that will be exclusive to Google’s Pixel devices.

9to5Google has a rebooted newsletter that highlights the biggest Google stories with added commentary and other tidbits. Sign up to get it early in your inbox, or continue reading 9to5Google Log Out below:

Besides launching alongside the Pixel 9, we know that:

“Pixie will use the information on a customer’s phone—including data from Google products like Maps and Gmail—to evolve into a far more personalized version of the Google Assistant.”
“…an AI assistant that aims to leapfrog assistants like Siri by carrying out complex and multimodal tasks, such as suggesting directions to the closest store where someone can buy a product they have photographed.”

There are still many unknowns about how Pixie will actually work to accomplish those goals. Keep in mind that the report came before the launch of the Gemini app (previously Assistant with Bard) in February.

Looking at the landscape, Apple is now rumored to “let users control individual app functions with their voice” using Sir in iOS 18i. According to Bloomberg, large language models will be leveraged:

That includes being able to open individual documents, moving a note to another folder, sending or deleting an email, opening a particular publication in Apple News, emailing a web link, or even asking the device for a summary of an article.

In the future, users will be able to string together multiple commands in one go.

That sounds a lot like the new Google Assistant that debuted on the Pixel 4. The premise in 2019 was that on-device voice processing would make “tapping to operate your phone would almost seem slow.” This system was aimed at letting you operate, multitask, and compose.

This next-generation Assistant will let you instantly operate your phone with your voice, multitask across apps, and complete complex actions, all with nearly zero latency.

The vision Google laid out five years ago is absolutely compatible with the one for AI agents offered at I/O 2024:

“I think about them as intelligent systems that show reasoning, planning, and memory. Are able to think multiple steps ahead, work across software and systems, all to get something done on your behalf, and, most importantly, under your supervision.”

Pixie could be the new Google Assistant done right now that the technology (LLMs) for it is ready. The version Google had in 2019 required users stick to certain phrasing rather than letting them speak naturally and then automatically discern the intention.

Google Research has been working on this with “Enabling Conversational Interaction with Mobile UI using Large Language Models.” Its LLM approach can answer questions about content that appeared in the UI, and control it after being given a natural language instruction.

Another big question is its relation to the Gemini app from a technical and branding level. What I foresee is Gemini, which is working to add more phone assistant capabilities, being available to all Android devices. Meanwhile, Gemini Live seems highly catered towards talking directly to Gemini rather than controlling your phone.

On Pixel, the Pixie initiative gives Gemini extra capabilities that let you use the phone with your voice. This allows Google to maintain one assistant infrastructure. Another thing to keep in mind is how the hardware and Android reorg saw the Google Research teams responsible for computational photography and on-device intelligence join the unified division to “bring deep AI expertise across platforms and devices.”

I don’t quite see “Pixie” (or some other name) as being its own branded offering. Google ultimately needs to convey that Gemini is available everywhere. That said, I personally think the name is quite fitting and evokes all the right assistive connotations.