Cisco Talos, the Cisco cyber-intelligence division, warns about a worrying trend: cybercriminals are increasingly taking advantage of the great language models (LLMS) to automate and improve their cyber attacks. According to the investigation, the malicious actors are not only exploiting the public the services, they also use custom models and versions with Jailbreak ‘. These tools allow them to orchestrate convincing phishing campaigns and write complex malicious code.
Thanks to their ability to generate persuasive text, solve problems and write code, LLMs are gaining popularity in all sectors. According to Hugging Face, a platform that houses LLMS, there are currently more than 1.8 million available models. The majority are equipped with safeguards and incorporated limitations (‘railings and’ alignments’) to avoid criminal uses.
“To combat the improper use of large language models, organizations must adapt their security measures accordingly”highlights Ángel Ortiz, director of cybersecurity at Cisco Spain. “This implies monitoring AI -related traffic, detecting suspicious warnings and training employees to recognize phishing emails generated by AI. In addition, we strongly recommend working exclusively with confidence models and well -protected platforms”.
LLMS without restrictions
However, Cisco Talos has identified a significant number of LLMS without restrictions that allow cybercriminals to elaborate highly realistic phishing messages and fraudulent communications, often free of grammatical errors or suspicious phrases. This increases the probability that victims reveal personal or corporate information.
Cisco Talos warns about the increase in the use of LLMS, both legitimate and modified, to create and extend malware and perfect phishing campaigns
Examples of these models are Ollama and Whiterabbitneo, the latter promoted as a tool for cybersecurity operations both defensive and offensive. Cisco’s analysis also highlights the methods to eliminate integrated restrictions (alignments). Users can modify training data sets and adjust the base models to eliminate restrictions, thus facilitating improper use.
Personalized malicious llms
Some cyber-of theincuents have gone further developing their own LLMS and promoting them on the dark website. These malicious LLMS can create harmful software autonomously, such as ransomware, remote access Trojans, Shellcode and various scripts.
In addition, these malicious tools help generate phishing emails, destination pages and configuration files. You can also verify stolen credit card data, scan websites for vulnerabilities and devise new criminal strategies. Examples of this type of malicious applications are Ghostgpt, Wormgpt, DarkGPT, Darkestgpt and Fraudgpt. Talos has detected that Fraudgpt, in particular, is part of a broader scam campaign.
Legitimate LLMS abuse
Given the limited viability of the LLMS without restrictions and the high risk of fraud with malicious models, many cyber-declines choose to exploit legitimate models. These models offer a powerful platform, provided that the attackers can avoid integrated security measures.
The main barriers are the training guidelines and security measures that prevent responses to little ethical or illegal consultations. To overcome them, Cyber-Decoons use techniques such as immediate injection, which tries to make jailbreak to the models and avoid their limitations.
