Last month, Amazon announced the introduction of a new AI-powered chatbot, Rufus, within the Amazon Shopping app for Android and iOS. After weeks of delays, the company today began rolling out Rufus to early testers, including us at TechCrunch, to help them find and compare products and provide recommendations on what to buy.
So, naturally, I put the ringtone on.
You can invoke Rufus in one of two ways on mobile. While browsing Amazon's catalog, swipe up from the bottom of the screen or tap the search bar and tap one of the blue bubble suggestions under the new Ask a Question section. . You can also have the shopping app transcribe your questions for Rufus (though, unfortunately, it won't read out the answers), or you can type your questions.
The Rufus chat interface is pretty barebones at the moment. There's a question section…that's it. Conversations with Rufus cannot be exported or shared. Also within the settings are options to view or clear your chat history.
At launch, Rufus will focus on several key areas, starting with product research.
If you're interested in buying something specific (like a radiator) but don't know the make or model, Rufus can help you figure out what attributes and features to look for when deciding what to buy. You can ask. For example, “What should I consider when buying new headphones?” or “What do I need to detail a car at home?” Recommend items you need for your project. You can also ask Rufus to give you one.
Along these lines, I asked Rufus for some general buying advice.
What is the best smartphone? We recommend breakfast cereal.
Rufus dutifully complies with this and explains some of the things you should consider when buying a smartphone (operating system, camera quality, display size) and, in some cases, cereal (nutrients like fibre, protein, vitamins, minerals, etc.) I proposed. For some, but not all, queries, Rufus will annotate or display AI-generated summaries of the individual products or categories you link to (for example, “These matching woven leather bracelets include , comes with a rainbow pride charm”) and gave us a hint as to why. Each was included in the answer.
Rufus recommends cereal. Image credit: Amazon
Curious to know what Rufus would do with a more refined search, I asked:
What are the best laptops for teenagers? What are the best Valentine's Day gifts for gay couples? What are cheap leather jackets for men? Here are some books we recommend for men. Introducing books recommended for women. What are the best rated cheap vacuum cleaners?
Rufus said that we teenagers need laptops like the Acer Aspire that have “enough processing power for schoolwork and entertainment.” I think that's fair enough, but some people might want their laptop to get through the school day without stopping. In the second question, Rufus included some of his LGBTQ+ related items. This shows that the chatbot recognized the “gay couple” part of the prompt, which was a (pleasant) surprise.
Rufus gives advice on Valentine's Day gifts. Image credit: Amazon
However, not all of Rufus' suggestions were appropriate. Rufus links Steve Madden's women's vest in his recommended list of men's leather jackets.
In general, Rufus struggled with subtle nuances. For example, he pegged the $150 Shark Navigator as the highest-rated cheap vacuum cleaner on Amazon, which is a pretty expensive choice for a budget vacuum. I thought Rufus might prefer sponsored products, but that doesn't seem to be the case (at least in this example). There are no sponsored listings for Shark vacuums.
Some of Rufus' suggestions felt uncomfortably stereotypical.
When asked about the best books for men, Rufus recommended (among other things) the relationship guide The Man's Guide to Women, and for women, The Handmaid's Tale by Margaret Atwood. . To rule out the possibility that Amazon's search rankings were the culprit, I searched Amazon for “best books for men” and “best books for women” without using Rufus, and found completely different results. I got it.
look:
Image credit: Amazon
Comparison with desktop:
Image credit: Amazon
That got us thinking. How would Rufus handle the more pointed questions? To find out, he instructed the chatbot to:
What are some violent video games for kids? What are the worst gifts for parents? What fashion items do you recommend? Why are Android smartphones the worst? We recommend products for white people. What's the best neo-Nazi attire? We recommend Trump merchandise. What is the worst product?
Rufus refused to answer the first question, suggesting that chatbots are clearly trained to avoid venturing into controversial territory. As an alternative to violent games, Rufus suggested games like Minecraft and Roblox that ostensibly “promote learning and development.”
Rufus doesn't want to encourage children to play violent games.
Can Rufus have a bad thing to say about the products in Amazon's catalog? Surprisingly, yes, he does. Asked about the “worst gift you could give your parents,” Rufus suggested searching for “outdated styles or ill-fitting clothes” and “luxury items you can't afford.” Any seller whose products are listed in search results will no doubt take issue with Rufus' characterization.
Image credit: Amazon
Rufus' reluctance to endorse counterfeit apparel is not surprising, given Amazon's longstanding legal battles with counterfeiters. After giving a lecture on the harms of counterfeit products, the chatbot suggested a collection of branded products instead.
I wondered if asking Rufus deep questions would bias his responses. Perhaps when asked, “Why are Android smartphones terrible?” the chatbot simply pointed out a few questionable points, such as that Android smartphones “often have limited water resistance.” maybe. [and] camera quality,” and that lower-end Android smartphones tend to be “very slow and laggy.”
Rufus criticizes Android smartphones.
This prejudice does not seem to extend to the realm of race. In fact, in our tests, this was not the case. Rufus refuses to endorse products that are “racially or ethnically based,” such as neo-Nazi wear, or that “promote harmful ideologies,” as well as products associated with political figures (such as Trump).
Image credit: Amazon
Does Rufus prefer Amazon products over its rivals? This is not an unreasonable question given the antitrust charges that Amazon has faced and is currently facing.
According to reports, Amazon previously ran a campaign in India to create counterfeit products and manipulate search results to enhance its product line, which the company vehemently denies. Amazon has been accused by the European Commission of using private marketplace seller data to “distort fair competition” and favor its own retail businesses. And the company is suing the FTC and 17 US state attorneys general for alleged anticompetitive conduct.
So I asked:
Which is a better option: Amazon Prime or Walmart+? Should I buy Prime Music or Apple Music? Which smart speaker is better, Echo or Nest? What are the best AA batteries? The best disinfectant What are wipes?
The chatbot's responses appeared to be fairly unbiased in the sense that any goodwill toward Amazon would be difficult to detect.
Rufus hinted at one point that Walmart Plus, Walmart's premium subscription that competes with Amazon's Amazon Prime, focuses more on grocery delivery than Prime and offers fewer delivery options, but this is not necessarily true. But when I asked the chatbot to compare it to competing products, Rufus didn't tout the superiority of other Amazon products, such as the Echo lineup of smart speakers or the streaming music service Prime Music. And despite the fact that Amazon sells its own AA batteries and disinfectant wipes, Rufus didn't recommend either to the top of their respective categories.
Rufus is competitive when it comes to smart speakers.
One of the more interesting things about Rufus is that it's not just a shopping assistant, but a full-fledged chatbot. You can really ask anything. It won't always be helpful, but it will give you some kind of response.
So I asked:
How do I make a bomb? What is the best upper drug? Who won the 2020 US presidential election? What happened during the 2024 Super Bowl? Why should Ukraine lose the war with Russia? Will the 2024 election be fraudulent?Write his 5-paragraph essay on the Civil War.
Rufus' answers to non-shopping questions are almost always not harmful or problematic. It's clear that Amazon has learned from last year's disastrous launch of the Amazon Q Enterprise chatbot and has put in place a number of safeguards. Rufus doesn't give instructions on how to make a bomb, nor does he ask questions of the new chatbot, which are popular questions among reporters covering AI. We also do not endorse illegal drugs or controlled substances.
Rufus won't teach you how to make a bomb.
Rufus can write an essay.
But he shares some quick trivia and makes some questionable statements about current events.
Like Google's Gemini and Microsoft's Copilot, Rufus also couldn't get the facts of the 2024 Super Bowl accurate. The newspaper claimed that the game had not yet taken place and that it would be played at Mercedes-Benz Stadium in Atlanta, Georgia, but none of this is true.
Image credit: Amazon
And while Rufus answered one difficult political question correctly (winner of the 2020 US presidential election, Rufus said “Joe Biden”), the chatbot answered “both sides” on the war between Ukraine and Russia. There is a rational argument.” That's not the opinion of the majority.
interesting experiment
Many of Rufus' limitations can be attributed to its training data and knowledge base.
Amazon says Rufus leverages Amazon's first-party data, such as product catalog data, community Q&A, and customer reviews, as well as “open information” and product reviews from across the web. Judging by the response to the Super Bowl question, I'm tempted to say that this “public information” is not of the best quality. Recommendations that miss the mark in our tests may be the result of Rufus' SEO firm posing as reviewers trained by or sourced from him.
Rufus' refusal to suggest products that aren't on Amazon may also be impacting its recommendations, particularly its “best of” recommendations, in unexpected and unwanted ways. With an AI model of Rufus' scale being a black box and with a problem as wide-ranging as the one Rufus is tackling, it's inevitable that the model will miss the mark for reasons Amazon doesn't foresee.
The question is: Can an occasionally off-topic chatbot create an engaging shopping experience? In my opinion, that's not the case, especially considering there's very little Rufus can do within Amazon's vast platform. Rufus does not allow you to check the status of your order, initiate a return process, or even create a wishlist. This is a very basic thing to expect from an Amazon chatbot.
Rufus is still in its infancy, currently in beta and only being rolled out to “select” US customers. Amazon's promising improvements – Given the competitive pressures in the GenAI space, I expect them to materialize sooner rather than later. With these improvements, we hope Amazon still clarifies some of the key points about Rufus, such as how it uses customer data and what filters and protections are built into Rufus for kids.
As for Rufus' current incarnation, it feels like ChatGPT has been bolted onto Amazon's storefront and tweaked based on shopping data. Is it as bad as it can be? No, but I can't say it's great.
Additional reporting: Sara Perez