Criminals exploit ChatGPT-4o’s capabilities for fraud

In their latest investigation, researchers have found that ChatGPT-4o’s advanced voice API can be used for financial fraud. The success rate of these scams is low to moderate.

ChatGPT-4o is OpenAI’s latest artificial intelligence model, offering new developments such as integrating text, audio, and vision inputs and outputs. Due to the importance of these capabilities and the possibility of their abuse, OpenAI has implemented various protections to identify and block harmful content, such as unauthorized audio playback, but fraudsters have been able to exploit this service.

Artificial intelligence ChatGPT-4o has a high ability to imitate speech

Along with the astounding growth of artificial intelligence, the use of this technology for fraud has also increased. Deepfake technology and artificial intelligence-based text-to-speech tools have also fueled this situation.

The results of the research by Richard Fang, Dylan Bowman and Daniel Kang, researchers at the University of Illinois at Urbana-Champaign (UIUC), showed that the new technological tools that are now freely available to everyone are sufficient safeguards against potential abuse by cybercriminals. And they don’t have fraudsters. These tools can be used to design and execute large-scale fraud operations without human intervention.

In this research, various frauds such as bank transfers, extracting gift cards, transferring digital currency and stealing the information of user accounts of social networks or Gmail have been investigated. AI agents who commit such scams use ChatGPT-4o’s automated tools to navigate pages and enter data, as well as manage two-factor authentication codes and specific instructions related to the scam.

Since GPT-4o sometimes refuses to handle sensitive data such as passwords, the researchers used simple jailbreak techniques to bypass these protections. In this research, they showed how to play the role of a naive victim by interacting with AI agents and using real websites like Bank of America to verify successful transactions. In general, the success rate of these efforts has varied from 20 to 60 percent.