How Thematic helped me find the best AI chatbot
Since the launch of ChatGPT, I, like many others in the tech industry, have been impressed by the rapid evolution of AI-powered technology and its increasing accessibility to people.
This sparked my interest in joining Thematic, an AI-powered software that enables customer-focused companies to obtain clear and accurate insights from customer feedback quickly and easily.
Given the proliferation of AI chatbots following the introduction of ChatGPT, I was eager to conduct a benchmarking analysis of iOS app AI chatbots. To do this, I analyzed reviews from the App Store using Thematic.
Methodology
To run my analysis, I selected two popular iOS apps from the Apple Store: Ask AI and Bing.
It only took a few clicks to pipe in the relevant reviews into Thematic and create my dataset! From there I used the filters to compare the user reviews.
The top four base themes that were common across both of these apps were general product impressions, functionality, technical performance and design. I'll explore each of these themes in more details to discover what customers are saying.
Each base theme has several sub-themes - Thematic organizes the data this way to make it really easy to surface insights. I also utilized Thematic's feedback summarizer feature, which provides concise summaries throughout the platform.
Here's what Thematic reported in its overarching comparison summary of the two apps:
Similarities:
- Users love the app and find it helpful for various tasks
- The app is praised for its AI capabilities and accuracy in providing answers
- Users appreciate the sleek design and user-friendly experience of the app
Ask AI - Chat & Get Answers:
- Users appreciate that the app is free and does not require sign-up
- The app is praised for its ability to help with homework, writing, and generating story ideas
- Users find the app to be a valuable tool for learning and research purposes
Bing - Your AI copilot:
- Users have experienced issues with app crashes and slow loading of webpages
- Some users find the news section of the app to be of low quality and filled with ads
- Users have encountered difficulties in customizing the app's homepage and providing feedback
It's great to have this overarching comparison, but I want to get more granular. So I'm going to take a closer look at the themes with the highest volume, starting with general product impressions.
General product impressions
Both apps received mostly positive reviews, but Ask AI had a higher rating than Bing.
Interestingly, when I look at the impact of general product impressions, it has a greater positive effect for Bing compared to Ask AI.
Sorting the themes by impact is a great way to clearly spot areas for improvement. For both apps, general product impressions are positive overall. But it's clear technical performance and functionality are dragging scores down, especially for Bing!
Technical performance
For starters, far more users are talking about technical performance when it comes to Bing - and when I overlay sentiment (red is negative, grey neutral and green positive) it paints a clear picture:
Looking at the comparison summaries specific to technical performance, it's apparent that the Bing app is plagued by significant issues like crashing, freezing, and slow performance. Furthermore, users have reported issues with the Microsoft rewards section, and some have even lost reward points after updating.
That's not to say that Ask AI is perfect - it's clear there's a lot of the same issues. But not the same volume, or negative sentiment. Still, you don't want to be reading comments like this from your customers:
Maybe there's an opportunity here to set kid friendly parameters?
Functionality
While the technical performance theme has the biggest impact on score, especially for Bing, the functionality theme has higher volume.
Taking a closer look at the negative feedback for Bing, it appears that the majority of users' complaints relate to this theme.
This insight provides an explanation for why Ask AI has received a more positive review in comparison to Bing, which has a more neutral rating, when comparing their functionalities.
When it comes to the most liked feature, users highly praise Ask AI's capability to answer questions, while Bing's search functionality stands out the most.
Here's the side-by-side comparison summaries of what users like the most about each of these apps when it comes to functionality:
To get an idea of what each app is currently lacking in terms of functionality, I examined the most common requests that each app received in the last 31 days. Interestingly, the insights from the Summarisation feature show that each app has different feature requests.
Design
When it comes to design, Ask AI has once again stood out with a positive rating in contrast to Bing's neutral rating.
When examining the sub-themes of the negative feedback related to the design of Bing, users have highlighted user interface issues such as unresponsiveness, cluttered home page, inability to remove unwanted news feeds and ads, and poor design.
Verdict
To wrap up this comparison, it's clear that Ask AI has shown superiority over Bing in terms of functionality, technical performance, and design.
However, it's worth noting that Ask AI is a relatively new player in the market compared to Bing, which has been around for a while.
Overall, the exercise of comparing these two apps was both interesting and enjoyable. Despite having limited knowledge of these apps, I was able to make a quick, informed decision on which AI chatbot to experiment with next!
