NewtonX research finds reliability is the determining factor when buying AI, but is brand awareness coloring perceptions?

June 12, 2024

Our research leaves no doubt. When decision-makers are debating which generative AI tools to invest in, it's reliability that tips the scales. But, whether this attribute is being evaluated on objective criteria is a different matter.

Get the download from Winston Ford, NewtonX Senior Product Manager:

Rank these eight key decision-making criteria, with 10 being the most important and 1 being the least important.

When we asked the 240 executives who took part in our NewtonX Prime Generative AI survey to do this, one criterion stood head and shoulders above the rest. 52.1% ranked reliability as their chosen generative AI tool’s most important attribute. 

In comparison, functionality and value only got 39.2% and 36.7% of the vote respectively. And, pricing, scalability, and the vendor’s reputation were at the bottom of the list, ranked most important by only 23.8%, 22.1%, and 20.8% of respondents respectively. 

Given the risks should AI malfunction —  Zillow, for instance, lost $8.3 billion because its algorithm was overvaluing properties — it’s not surprising that decision-makers would rate reliability so highly. 

But the results beg two important questions. How do you objectively evaluate a generative AI tool’s reliability? And how do you determine whether it meets an acceptable standard?

Data scientists typically use mathematical formulas such as F1 score to measure AI models’ performance. But our research suggests the less objective metric of brand awareness may also be having an impact in the boardroom. 

ChatGPT, currently used by 81.3% of our survey participants, was ranked 7 or above for reliability by an overwhelming majority: 80.4% of respondents.

By contrast, the lower the awareness of a generative AI tool was among our survey participants, the less likely they were to rate it highly for reliability. Generative AI tools with low awareness also have negative net promoter scores.

Of course, this might change as awareness increases. But that begs two other questions. Will newer generative AI tools ever be able to challenge ChatGPT’s ubiquity? And what will it take to do so?

Sign up for our newsletter, NewtonX Insights:

Your playbook to making confident business decisions enabled by B2B research. Expect market research trends, tools, and case studies with leading enterprises, delivered monthly.

Related Content

How The Wall Street Journal cut through the generative AI haze with NewtonX insights

How do marketers plan to realize gains from AI? The Wall Street Journal partnered with NewtonX to survey 82 executives for the scoop.

read more

The right brand perception survey questions boost your marketing ROI

In B2B branding, brand perceptions are what your target audience, customers, and prospects—and to some extent, your employees—think about your brand.

read more

How to use generative AI deliberately, for better B2B research results

Originally published on Greenbook on May 15, 2023.    Walk into any research forum—or an IIEX conference—and you’ll find the conversation will invariably turn to AI. Generative AI may be all the buzz, but it’s

read more

Despite skyrocketing demand, only 10% of AI initiatives are delivering substantial ROI

Despite skyrocketing demand, only 10% of AI initiatives are delivering substantial ROI. It will take hard work, dedication, and a whole lot of resources to get where you want to be.

read more

ChatGPT dominates the AI market, but the Google threat looms

With sky-high brand awareness, a reputation for reliability, and a positive NPS, ChatGPT seems untouchable. But our research suggests business executives aren't so sure.

read more