Dangers for Companies – How Cybercriminals Exploit ChatGPT
Dangers for Companies – How Cybercriminals Exploit ChatGPT
AI applications are used by many Germans. Companies also deploy them to quickly generate codes using ChatGPT, for example. However, cybercriminals exploit the program’s susceptibility to errors. What companies need to consider when dealing with ChatGPT and similar tools.
Applications based on artificial intelligence have become mainstream. According to the “Opinion Monitor on Artificial Intelligence,” around half of Germans have a positive attitude towards AI programs like ChatGPT. Approximately one-fifth of Germans have already used the text generation software, according to Bitkom. However, 44 percent also fear AI and its impacts. Indeed, while AI applications can be very useful, they also increase the cybersecurity risks for companies.
Cybercriminals place malicious code in ChatGPT
ChatGPT is not only capable of writing a birthday speech or summarizing a scientific article but can also be used to create codes. The problem: ChatGPT still tends to respond to requests with made-up answers, citations, and directories. Cybercriminals exploit this flaw to place malicious code at these invented locations by ChatGPT.
If companies access these sources and download the corresponding codes, they significantly endanger the security of their systems. The rule of thumb is therefore: never download and execute unchecked codes from ChatGPT! Every generated line of code must be tested in a secure environment, and regular, protected backups of the original system codes are essential.
Using AI in Companies – What to Consider
To avoid creating additional risks when working with AI programs, companies should keep some fundamental rules in mind. These include:
- Risk awareness: Careless handling of ChatGPT and similar tools can quickly become dangerous. It is important to raise awareness of the negative impacts that AI can have.
- Data sensitivity: Employees should not feed AI programs with sensitive data such as passwords or personal information.
- Proven security practices: What applies to all areas also applies especially to handling AI: employees must apply proven security practices, change their passwords regularly, and protect sensitive information.
- Know the limits: Companies must be aware that ChatGPT is still not a mature program and therefore regularly reaches its limits. Its understanding is still very limited, so all outputs of the program must be thoroughly checked.
AI as a Security Assistant
At the same time, AI applications like ChatGPT can be more than useful helpers in defending against cyber threats. For example, the program can be used to check software for security vulnerabilities. The AI only needs to be trained with a few data points and relieves human employees, who can thus perform security checks without understanding the basic mechanisms of the software being checked. Additionally, ChatGPT can be used as a spam filter. The AI is much more efficient and accurate than a conventional spam program.
ChatGPT can thus primarily relieve its human colleagues, who are reaching their limits due to the increase in cybersecurity incidents and the flood of security-relevant messages that need to be processed daily. The AI performs an initial evaluation and can carry out basic checks, while humans focus on the truly important events. Company decision-makers can thus benefit from the advantages of AI applications like ChatGPT and simultaneously minimize the risks to the security of their own systems.
TL;DR
- Cybercriminals place malicious code at sources and directories invented by ChatGPT
- Companies must never execute AI-generated code without checking – testing in a secure environment is mandatory
- ChatGPT can also be used as a security assistant: for code audits, spam filtering, and incident analysis
Key Facts
AI Usage in Germany: Around one-fifth of Germans have already used ChatGPT (Bitkom)
Main Risk: ChatGPT invents citations and directories – attackers place malicious code there
Security Rule No. 1: Never download and execute unchecked codes from ChatGPT
AI as Defender: ChatGPT can be used as a spam filter and for automated security checks
Fact: Deepfake-based social engineering attacks increased by 550 percent in 2024, according to Europol.
Fact: AI-powered cyberattacks increased by 135 percent in 2024, according to Darktrace.
Frequently Asked Questions
Every question is locked. A tap unlocks the answer.
How do cybercriminals use ChatGPT for attacks?
ChatGPT tends to invent non-existent sources and package directories. Attackers register these invented sources and place malware there. If developers download the code recommended by ChatGPT, they infect their systems.
What rules should companies establish for AI use?
Companies need clear guidelines: do not feed the AI with sensitive data, do not execute generated code without checking, change passwords regularly, and have a general awareness of the technology’s susceptibility to errors.
Can ChatGPT also improve cybersecurity?
Yes. ChatGPT can check software for security vulnerabilities, act as an intelligent spam filter, and perform an initial evaluation of security-relevant messages. This relieves IT security teams in their daily incident analysis.
Should employees enter sensitive company data into ChatGPT?
No. Employees should not enter passwords, personal information, or confidential business data into AI systems. The entered data could be used for training or become accessible through security vulnerabilities.
How can AI-generated code be used safely?
Every line of code generated by ChatGPT must be tested in an isolated sandbox environment. Additionally, regular, protected backups of the original system codes should be created to enable quick restoration in case of damage.
Further Reading in the Network
AI Security in Cloud Environments on cloudmagazin.com
AI Use in Daily Business Operations on mybusinessfuture.com
Responsible AI Use as a Leadership Task on digital-chiefs.de
Bild: Matheus Bertelli/pexels.com https://www.pexels.com/de-de/foto/frau-laptop-arbeiten-internet-16094040/
Related Articles
- Cybersecurity Trends 2026: The 7 Developments Security Decision-Makers Need to Know
- Hybrid Warfare and Disinformation: The Underestimated Cyber Threat to Companies
- Palantir and the Future of Cyber Defense: AI as a Strategic Weapon
Header Image Source: Pexels
