Amazon’s AI shopping assistant, Rufus, is now making product suggestions. The problem is that it is often wrong.
Rufus has been available to all U.S. customers since July and beta testers since February. The assistant is prominently visible in the app as a button in the bottom right, showing Amazon’s commitment to the new feature.
Recently, it got more confident and started answering questions with product recommendations. It now shows a specific product rather than pointing to a shelf. Rufus responds to questions with five products that shoppers can buy with two clicks (once on the product tile and once on the buy button). Before, it would answer with links to search results from which the shopper would have to pick the product, sorting through dozens of options and taking at least one extra click to buy.
However, all AI systems suffer from hallucinations, and they are glaringly obvious in Rufus. If asked for the cheapest options, it will respond with items that are not the cheapest. If asked for the best marathon running shoes, it will respond with options no runner or coach would recommend. If asked for TV recommendations for gaming, it will include options that are not TVs. And if asked for winter gloves, it will respond with a list seemingly picked at random.
Rufus is more confident but not more intelligent. Amazon is working to fix that, and they are not alone. In November, AdWeek reported that Perplexity is adding “Buy with Pro” buttons to shopping results, allowing customers to complete transactions without leaving the platform. In October, Google rolled out changes to Google Shopping, which uses AI to recommend products, summarize reviews, and suggest relevant articles.
The promise of AI is analyzing the more than 40,000 cycling gloves in Amazon’s catalog to save shoppers from having to click around, watch videos, and read reviews. Once Amazon adds a buy button to the AI response, buying a product will take just one click. And then, in the future, it will take no clicks at all once “Rufus, buy the best cycling gloves for winter” can take action. However, that final step might not happen, as, above all, Amazon’s business is to sell ads. It started testing ads in Rufus in September.
If AI interfaces replace search — a distant and seemingly impossible yet already slowly happening future — the Amazon and Google algorithms gain even more power. Power to decide what five products shoppers see or which one product they default to. But as they gain power, sellers and brands lose footing. The game as old as the internet, known as SEO, will be replaced with making the black-box AI algorithm see. AI’s Choice is the new Buy Box.