International Business Machines fell short in its latest attempt to prove machines can triumph over man. But it came close.
The tech giant’s six-year-old artificial intelligence debating system, affectionately dubbed “Miss Debater,” went head-to-head with one of the world’s most decorated practitioners Monday. After a 25-minute rapid-fire exchange about pre-school subsidies—during which the female-voiced AI showed flashes of very homo sapien humor—the audience handed the victory to 31-year-old Harish Natarajan.
The unorthodox contest marked the latest highly marketed man-versus-machine challenge. In 1996, IBM created a computer system that beat a chess grandmaster for the first time. In 2011, its Watson supercomputer defeated two record-winning Jeopardy! contestants. And Alphabet AlphaGo famously proved AI can master the ancient and intricate game of Go. But debating—which requires creativity and emotive elocution—has proven more elusive.
IBM’s machine—known formally as Project Debater—kicked off Monday’s match-up with a cheeky greeting. “I have heard you hold the world record in debate competition wins against humans, but I suspect you have never debated a machine. Welcome to the future.”
The event unfolded in front of hundreds of journalists, tech industry insiders, and software engineers at IBM’s Think conference in downtown San Francisco. The topic: We should subsidize preschools. While spectators voted Natarajan the victor they also said IBM’s machine better enriched their knowledge.
Both contestants were given the topic at the same time and had 15 minutes to pare down arguments into a four-minute speech, four-minute rebuttal, and two-minute summary. Standing at human height, Project Debater’s ominous black box remained silent except for three rotating blue circles as it mulled over 10 billion sentences from news articles and scientific journals. Facing her onstage, Natarajan scrawled notes on scrap paper.
While the AI lost, the event was a culmination of sorts for the project’s progenitor Noam Slonim. Sitting in the front row on Monday night, the IBM researcher could be seen laughing and cringing during the proceedings. He knew they were the underdogs: Natarajan holds the world record for most debate competition victories and has attended three world championships, winning the European tournament in 2012.
“It’s like you’re sitting there in the audience with your kid on stage competing against a world-class pianist and everybody is watching,” he said. Unlike chess or Jeopardy, debating requires connecting with people and convincing them of a viewpoint. Delivery is key and “this is human territory.”
Project Debater used research and quotes from politicians to support her argument that subsidizing preschools isn’t just a matter of finance, but a moral and political duty to protect some of society’s most vulnerable children. Natarajan countered that, too often, subsidies function as politically motivated giveaways to the middle class.
The biggest advantage any human holds over Project Debater is the ability to deliver speech with emotion, wielding tone, inflection, pitch, and pauses to sway an audience. A week ago in London, Natarajan predicted he might have the edge. “I imagine at this stage a human would still find it easier to construct logical arguments than a machine would in a way which is reasonably convincing to a human audience,” he said at the time.
But the track record of humans vanquished by AI played on Natarajan’s mind in the lead-up to the debate. He’d watched AlphaGO, a documentary about the contest between AI and the world’s top Go player, who was overconfident and wound up losing four games to one. Natarajan realized “the hubris with which humans sometimes take playing against a machine.”
Slonim hatched the idea of Project Debater in 2011 while Watson’s victory in Jeopardy still resonated. The following year, he led a research team in Israel that began studying the way humans learn the art of debate, and built a system to mirror that process. Their machine scans more than 300 million newspaper articles and scientific journals to identify relevant arguments on any given topic. It then has to determine which facts and opinions are for or against, build a speech, and deliver it in a clear and cohesive manner. What’s more, the AI then has to comprehend her opponent’s argument and craft a rebuttal.
Project Debater gets us closer to realizing the dream of AI pioneer and British mathematician Alan Turing. In 1950, the founder of modern computing raised the question of whether or not machines could think for themselves. He predicted one day we would hold a conversation with a computer and not be able to tell the difference between human and machine.
Miss Debater still needs some work. Longer-term, Slonim and his fellow researchers, Ranit Aharonov and Talia Gershon, are exploring whether AI can help expand the human mind. For instance, Project Debater may one day help lawyers pore through thousands of court cases to form closing arguments, or help children develop critical thinking skills.
“Think about it for a moment. We don’t often see a machine having an intelligent discussion with a human for 25 minutes,” Slonim said.
IBM’s shares are up 20% this year.
