Intercom runs customer service solutions, which have been supercharged by the onset of large language models (LLMs). 

Since the advent of high-skilled LLMs, companies like Intercom have argued that customer service work is repetitive and will inevitably sap employees’ morale. Developing automated solutions to combat this, however, is tricky, given the vagueness or ambiguity of customer queries. 

Then there’s the question of accuracy, groundedness, and ensuring that an AI-powered chatbot is actually answering the questions posed by its human subjects.

Despite these challenges, Intercom is seeing early signs of success.

Des Traynor, Intercom’s cofounder and chief strategy officer, told Fortune his company has been able to double the amount of support volume it handles while keeping its staff size the same since integrating generative AI into customer service queries. 

Intercom CEO Eoghan McCabe announced in May that the company was pumping an extra $100 million of investment into AI capabilities and adding 75 new jobs, with a focus on expanding its machine learning division.

Part of that investment involves partnering with Anthropic’s Claude to power its AI-driven Fin 2 customer service bot rather than OpenAI’s ChatGPT, which powered the original Fin.

The diversion away from its use of ChatGPT isn’t down to a specific problem, Traynor says, but comes from the result of what he calls a “Torture Test,” which Anthropic’s Claude passed with flying colors.

Fin 2 has been able to achieve an average resolution rate of 51% with an answer accuracy of 99.9% among its thousands of Intercom users by switching its AI engine to Claude.

“It’s not like there’s a gotcha example that one LLM doesn’t perform on and Anthropic does,” Traynor told Fortune.

“We look at millions of answers—who had the highest percentage resolution? And that was the reason why we partnered with Claude.”

Anthropic’s CEO quit OpenAI to set up the company in 2021 in hopes of building a more trusted model that had a focus on safety. 

The group championed the use of “constitutional AI,” with its bots following a set of principles to allow for more transparency.

Mike Krieger, cofounder of Instagram and chief product officer at Anthropic, described the Intercom linkup to Fortune as his “favorite kind of partnership” because it’s based on “experimentation and evaluation.”

Anthropic applied these values to Intercom’s chat function.

“Does it have a sort of a humanistic view of how it talks to people? Is it reasoning about things in a complex way, in a way that is going to be able to solve their problems? Does it not hallucinate?” 

Anthropic has also deployed Fin 2 in its channels, in what Intercom’s Traynor was wary of describing as a “love-in” between the companies. Still, Anthropic’s Krieger hailed the fact that his company reached zero ticket queries through its use of the tech.

“The team was relieved, excited, and just very impressed with what we’ve seen already,” he says.

Customer service has been one of the early winners in the generative AI revolution, with companies in customer-facing sectors hailing the enhanced speed and productivity in dealing with queries with the help of LLMs.

Swedish buy-now, pay-later giant Klarna has said that its AI chatbot, powered by OpenAI, is doing the work of 700 customer service agents. Klarna CEO Sebastian Siemiatkowski intends to trim his workforce in the coming years as a result of the company’s use of AI.

Traynor says there are “different-shaped people” joining his company since introducing the original Fin, including the hiring of “conversation experts” to control the flow of customer service interactions.

“We’re hiring a lot of PhDs —±… Experimentation is now like a first-class citizen in our culture,” said Traynor.