UK study warns of perils in AI-driven 'intention economy'

LONDON

Conversational artificial intelligence (AI) tools may soon "covertly influence" users' decision making in a new commercial frontier called the "intention economy," University of Cambridge researchers warned in a paper published yesterday.

The research argues the potentially "lucrative yet troubling" marketplace emerging for "digital signals of intent" could, in the near future, influence everything from buying movie tickets to voting for political candidates.

Our increasing familiarity with chatbots, digital tutors and other so-called "anthropomorphic" AI agents is helping enable this new array of "persuasive technologies," it added.

Left unchecked, that could allow for "social manipulation on an industrial scale," the pair argued in the paper.

It characterises how this emergent sector, dubbed the "intention economy," will profile users' attention and communicative styles and connect them to patterns of behavior and choices they make.

"AI tools are already being developed to elicit, infer, collect, record, understand, forecast, and ultimately manipulate and commodify human plans and purposes," co-author Yaqub Chaudhary said.

The new AI will rely on so-called Large Language Models, or LLMs, to target a user's cadence, politics, vocabulary, age, gender, online history, and even preferences for flattery and ingratiation, according to the research.

Co-author Jonnie Penn warned: "Unless regulated, the intention economy will treat your motivations as the new currency."

"It will be a gold rush for those who target, steer, and sell human intentions," he added.