Tuesday, December 3, 2024

The race is on to make AI agents do your online shopping for you | TechCrunch

Must read

Millions of Americans will pop open their laptops to buy gifts this holiday season, but tech companies are racing to turn the job of online shopping over to AI agents instead.

Perplexity recently released an AI shopping agent for its paying customers in the United States. It’s supposed to navigate retail websites for you, find the products you’re looking for, and even click the checkout button on your behalf.

Perplexity may be the first major AI startup to offer this, but others have been exploring the space for a while — so expect to see more AI shopping agents in 2025. OpenAI and Google are reportedly developing their own AI agents that can make purchases, such as booking flights and hotels. It would also make sense for Amazon, where millions of people already search for products, to evolve its AI chatbot, Rufus, to help with checkout as well.

Tech companies are using a mix of new and old techniques to get around the barriers erected by retailers to block unwanted bots from using their sites. Rabbit released its LAM Playground earlier this month, which lets an AI agent navigate websites on your behalf using a computer in a data center. Anthropic’s computer use agent does the same thing, but it’s hosted on your personal computer.

Meanwhile, Perplexity is partnering with Stripe to leverage some older payments features that have been repurposed for AI agents.

Stripe is allotting single-use debit cards for Perplexity’s AI agent to spend money with online — a repurposed version of the Stripe Issuing feature. This makes it so the agent can buy you a pair of socks without needing access to your entire bank account. That way, if it hallucinates, the agent just buys the wrong socks for a few bucks, and doesn’t spend your rent money on, well, socks.

Google’s AI agent reportedly needs access to your credit card information, which could give consumers pause. However, several companies already know your billing info – such as Google, Amazon, Apple, and Shopify – and they regularly fill out forms for you when you’re shopping online. This could give these companies an advantage when they ship products in the space.

These tools could reshape online shopping — something retailers and advertisers making a fortune from the status quo may not be happy about.

Just as AI chatbots have proven somewhat useful for surfacing information that’s hard to find through search engines, AI shopping agents have the potential to find products or deals that you might not otherwise have found on your own. In theory, these tools could save you hours when you need to book a cheap flight, or help you easily locate a good birthday present for your brother-in-law.

There’s a long way to go before AI agents can buy everything on your holiday wishlist, but there’s a lot of companies vying to do it.

Based on our early attempts, Perplexity’s shopping agent takes hours to process purchases, and sometimes runs into issues where it can’t purchase items at all. Overall, using the agent today seems more complicated than buying something on Amazon.

Perplexity also says there are human checkers involved to ensure its AI agent is working accurately. Having a “human in the loop” is not uncommon for the AI industry — but then, most AI chatbots don’t see the items I’m purchasing and my billing address. This raises some privacy issues for Perplexity, and whatever company is hiring its human checkers.

TechCrunch tested out Perplexity’s shopping agent by asking it to buy us toothpaste.

After prompting Perplexity with, “I’d like to buy toothpaste,” the agent returned several options from Walmart, Amazon, and some smaller websites. For a few options, Perplexity offers a button under the product called, “Buy with Pro” while other options take you straight to the website of the retailer. Buy with Pro is Perplexity’s shopping agent at work.

Prompting Perplexity’s Shopping agent (left), results (middle) and purchase confirmation (right) (Image Credit: Maxwell Zeff & Perplexity)

I chose a tube of Crest from Walmart. Without leaving the Perplexity app, I was able to checkout, and (seemingly) purchase the toothpaste. But instead of paying Walmart, my bank statement showed that I had paid Perplexity’s agent.

Three hours later, I received an email from Perplexity that its agent was not able to buy me the toothpaste, because it was sold out at Walmart. The next day, I tried to purchase another tube of Crest with Perplexity’s shopping agent. Eight hours later, I got a confirmation from Perplexity that it worked.

So what gives? Why did my first purchase get rejected, and why did both take hours to complete?

While Perplexity Shopping might seem a lot like Amazon or the TikTok Shop, where you can buy items from a wide array of merchants who upload and manage storefronts on the platform, it’s actually completely different.

Perplexity’s AI agent seems to be scraping retailers’ websites and giving you information about their products. Because this process isn’t necessarily real-time, it can cause a disconnect between what Perplexity tells you and what a store actually has in stock, which appears to be what happened in my case.

Perplexity declined to comment on whether retailers like Walmart were aware that their products were appearing on its app. This suggests that their scraping and purchase process is not authorized by those companies — something that could complicate buying or returning items.

You’re also not actually buying anything when you check out in Perplexity’s app. You’re paying Perplexity the exact amount that item costs, giving its AI agent instructions to buy a specific item, and telling it to fill out your name and shipping address in the process. Some time later, perhaps hours, the agent executes that task, or at least tries to.

“This is the equivalent of giving a small pot of money to an assistant in the real world, and give them rules about how they’re allowed to spend it,” said Stripe product lead Jeff Weinstein, who helped build Stripe’s AI agent toolkit, in an interview with TechCrunch.

But instead of giving money (in a pot or otherwise) to a real human assistant, who I would trust to buy toothpaste on their own, Perplexity’s AI agent occasionally needs to be monitored by another human. And even then, it doesn’t always work.

“I can’t disclose specifics around how Buy with Pro works, but what I can say is that there is human oversight providing occasional support, which ensures that transactions are completed in a timely manner and we avoid issues like purchasing the wrong product,” said Perplexity spokesperson Sara Platnick in an email to TechCrunch.

These days, hiring human checkers to watch AI systems is commonplace. Companies like Scale AI and Turing have built large businesses around the service. But in this case, Perplexity declined to answer TechCrunch’s questions around how often human oversight was necessary, how involved humans are in the process, and whether human checkers are watching AI agents make purchases in real-time. The lack of transparency here may not bother everyone, but it’s certainly worth noting.

If AI shopping agents really take off, it could mean less people going to online storefronts, where retailers have historically been able to upsell them or promote impulse purchases. It also means that advertisers may not get valuable information about shoppers, so they can be targeted with other products.

For that reason, those very advertisers and retailers are unlikely to let AI agents disrupt their industries without a fight. That’s part of why companies like Rabbit and Anthropic are training AI agents to use the ordinary user interface of a website. That is, the bot would use the site just like you do, clicking and typing in a browser in a way that’s largely indistinguishable from a real person. That way, there’s no need to ask permission to use an online service through a backend – permission that could be rescinded if you’re hurting their business.

Rabbit CEO Jesse Lyu said in a recent interview that AI agents are getting better than humans at solving CAPTCHA, the human verification tests that have previously prevented bots from shopping online. That means website owners will need to develop more sophisticated ways to prove personhood online.

It’s possible that one day, AI agents could be part of a better online shopping experience than what exists today. Perplexity’s shopping agent isn’t that by a long shot, but it offers an early glimpse of what could be.

In the next year, we’re likely to see better versions of AI shopping agents from Perplexity, OpenAI, and Google. We may just be seeing the tip of the iceberg in terms of how this could reshape the online retail industry, and what sorts of problems AI agent developers could run into.

Latest article