AI vs. AI: Technologies That Help Guard Against AI-Generated ScamsAI vs. AI: Technologies That Help Guard Against AI-Generated Scams

AI chatbots and rich communication services can protect business and personal transactions

James Stokes, Head of enterprise, U.K. and Nordics at Infobip

January 2, 2025

5 Min Read
A deepfake facial scan
Getty Images

While love might be priceless, it’s costing people millions. In the first half of 2024, more than £6.8 million was stolen from banking customers through romance scams. That’s a staggering 27% increase over the same period last year. Findings suggest that scammers are using AI to help them operate more sophisticated and convincing scams on a greater scale.

To illustrate the scale of the problem, an Opinium survey of 2,000 people for Santander in July found that nearly a third (29%) said they would offer financial help to a romantic partner they’d known for less than six months, yet nearly two-thirds (65%) believe that they would not themselves fall victim to a scam. With the amount defrauded now in the millions, the numbers would beg to differ.

The Impact of Catfishing

So, how do they do it? Scammers are harnessing natural language processing (NLP) tools, the technology that powers advanced AI models like ChatGPT, to create convincing personas that can interact and respond in real time. Known as catfishing, this tactic uses fake personas to lure victims into fictional relationships, gaining their confidence and, eventually, their money. These romance scams convince victims to send money, or personal information which the scammers then use to steal funds.

Related:Why Fintech’s Love Affair With Artificial Intelligence Needs a Reality Check

These capabilities have taken catfishing and fraud to the next level, able to replicate the tone, speech and emotions of real or fictional individuals, making them more persuasive than ever, without the fraudster even needing to be behind the keyboard. Add deepfake technology to the mix, generating fake video or audio that seems strikingly real—think “Fake DiCaprio” on TikTok—and the scam becomes all the more convincing.

And these threats aren’t limited to romance scams; advanced AI is being used in various types of fraud. With global daily screen time averaging more than six hours, all users are increasingly vulnerable to remotely driven fraud attempts.

For businesses, too, cybercriminals can impersonate high-ranking executives, tricking employees into sending cash or confidential information. They can also impersonate customer service representatives, convincing customers to hand over valuable personal data—also then damaging the company’s reputation.

Most would agree that as criminals' use of AI makes them able to blur the line between authentic and deceptive interactions, finding effective solutions is more crucial than ever.

Tackling the Threat

One promising tool for countering these threats is rich communication services (RCS).

Related:AI Could Ease Record Holiday Travel Disruptions

Initially designed as an SMS successor more than a decade ago, RCS now provides a richer messaging experience to over a billion users worldwide — and with Apple’s recent adoption of RCS on iOS 18 in September 2024, it’s now accessible to an additional 2.5 billion users.

RCS is a newer, better version of SMS that allows network operators and phone carriers to support media-rich content and provides extra layers of security and authenticity—a key feature as SMS is one of the main ways fraudsters initially contact their victims.

RCS Business Messaging, able to be used by brands and companies to communicate with their customers is modernizing carrier messaging, which was previously limited to SMS and in some markets MMS, by enabling businesses to send images, videos and interactive elements, to user’s default messaging inbox. RCS Business Messaging employs Transport Layer Security (TLS) encryption, which makes messages unreadable without a decryption key. And because it’s implemented via telecom providers, it’s subject to greater regulatory and legal frameworks and in turn more challenging for scammers to manipulate.

One particularly effective feature of RCS Business Messaging in preventing fraud is its verified sender profiles. Before being approved by telecoms to use the channel, businesses must complete a verification and approval process to use these branded profiles, which display a company logo, verified checkmark and direct links. This helps users recognize genuine messages from trusted sources, making them easier to distinguish them from fraudulent ones.

While RCS doesn’t yet allow for personal verification, it is tied to a specific phone number and device. Its ability to send high-quality media is also harder for fraudsters to replicate, even with AI advances—though not impossible. Additionally, RCS supports two-factor authentication, adding a layer of verification that can prevent unauthorized access.

The Road Ahead

Undoubtedly, RCS is a major step forward, but it’s not a foolproof solution. Scammers will always evolve and adapt their tools and techniques. A recent PwC report predicted that AI-driven fraud attempts will increase in both volume and sophistication — and urged businesses to explore more advanced detection methods.

PwC suggests fighting fire with fire, using either proprietary or third-party AI models to alert where patterns of communication with staff begin to look like social engineering attempts. This functionality may operate like a spam filter and alert compliance teams to conversations that appear to be of concern.

AI-driven chatbots can also provide 24/7 support for consumers who feel they may be at risk, or have been a victim of, fraud, assist with finance management and monitor for unusual transactions.

Whilst such tools can undoubtedly help, some common-sense practices can also make all the difference. Banks recommend always removing emotion from decision-making, being wary of pictures as they can be edited (use Google reverse image search) and beware of anyone talking online who will not video call or meet in person.

About the Author

James Stokes

Head of enterprise, U.K. and Nordics at Infobip , Infobip

James works with some of Infobip’s largest enterprise clients, helping them to build lasting customer relationships via personalized communications over any channel. James’ passion for CX can be traced back over 15 years with Virgin Mobile, Sky and OnDemand Group, and more recently head of account management at OpenMarket, now part of Infobip. James leverages his extensive knowledge and expertise in this space to drive growth and innovation for Infobip and its customers.

Outside of work, James is a keen traveller, golfer, and proud dad to two boys. James works with some of Infobip’s largest enterprise clients, helping them to build lasting customer relationships via personalized communications over any channel.

Sign Up for the Newsletter
The most up-to-date AI news and insights delivered right to your inbox!

You May Also Like