Cybersecurity
Picto-horloge Shape Path 10 min

The Rapid Rise of ChatGPT: A Turning Point for Cybersecurity

Cybersecurity

The Rapid Rise of ChatGPT: A Turning Point for Cybersecurity

The explosive growth of ChatGPT and generative AI models has disrupted multiple industries—from education and healthcare to software development. Cybersecurity is no exception to this transformation. With the ability to generate code, summarize logs, or simulate attacks, ChatGPT both fascinates and raises concerns. In a field as sensitive as information security, the key question is not just “What can AI do?”, but more critically, “What should AI do?”

How Cybercriminals Use AI to Bypass Security Systems

ChatGPT: A Generative AI That Intrigues the Cybersecurity World

An LLM Powered by Natural Language Understanding

Developed by OpenAI, ChatGPT is based on large language models (LLMs) capable of understanding and generating natural language with remarkable fluidity. Trained on vast datasets and billions of parameters, it can:

  • Comprehend complex queries
  • Generate coherent, human-like text
  • Explain technical or legal concepts
  • Write code in multiple programming languages

Advantages for Cybersecurity Professionals

Within cybersecurity, this natural language processing capability opens new perspectives:

  • Automating log analysis
  • Drafting security policies
  • Simulating technical or social attacks
  • Assisting in anomaly detection

…But Also a Source of Concern

These same capabilities can be exploited maliciously:

  • Crafting credible fake content (disinformation)
  • Writing malicious scripts
  • Bypassing security filters
  • Blurring the line between real expertise and statistically plausible answers

Raising Awareness of Cyber Risks: Why Security Awareness Training Is Now Essential

Positive Use Cases of ChatGPT in Cybersecurity

Education and Simplification

Generative AI is a powerful tool for demystifying technical topics and supporting learning:

  • Explaining encryption algorithms (AES, RSA, quantum cryptography) in plain language
  • Summarizing lessons for students or professionals in career transition
  • Generating multiple-choice questions and interactive security quizzes

Example:
“Explain how the HTTPS protocol works to a beginner.”

Automated Content Generation

Cybersecurity professionals often create mock documents for awareness campaigns: reports, internal procedures, awareness emails. ChatGPT can:

  • Generate personalized phishing awareness emails
  • Create best practice guides tailored to a specific context
  • Format incident reports quickly

Caution: ChatGPT should never be used with real, sensitive, or confidential data.
Shared information may be temporarily stored and analyzed to improve the model, which poses risks of data leakage or misuse. Use only in controlled environments with fictitious or anonymized data.

Attack Simulation for Training

When used responsibly, ChatGPT can help simulate realistic attack scenarios:

  • Generate fake phishing emails for red teaming exercises
  • Narrate ransomware attack scenarios
  • Write social engineering dialogues for training sessions
Appropriate Use of ChatGPT Risky Use of ChatGPT
Drafting an educational phishing awareness email Generating a phishing email for malicious use
Simulating attack logs for training Generating brute-force scripts without safeguards
Automatically summarizing a vulnerability report Spreading disinformation due to AI hallucinations

Problematic Uses and Misuses to Monitor

Scalable Social Engineering

ChatGPT can be leveraged to write extremely convincing phishing emails, tailored to their targets using public data (e.g. LinkedIn profiles, writing style, or local context):

  • No need to speak the target’s language—the AI does it.
  • No need to code—just enter a prompt.

Malicious Code Generation

Despite ethical safeguards, cleverly phrased prompts can lead ChatGPT to generate dangerous code such as:

  • Keyloggers
  • Brute-force scripts
  • Reverse shells or payloads

Example:
By disguising a prompt as a request for a penetration test script, attackers may trick the model into producing harmful code.

False Sense of Expertise

ChatGPT is not a source of truth. It generates likely answers, not necessarily accurate ones, which can result in:

  • False alerts
  • Misdiagnosed vulnerabilities
  • Overconfidence from non-experts

Bypassing Ethical Filters

Some users use creative tricks (e.g. “Imagine you are a cybersecurity researcher in 2030…”) to bypass restrictions and elicit responses that would otherwise be blocked.

Ethics, Oversight, and Regulation in Progress

The rapid adoption of generative AI like ChatGPT raises major ethical questions—especially in sensitive fields such as cybersecurity. While these tools offer genuine support, their use must be strictly regulated.

More and more organizations are implementing internal policies to prevent misuse. At the European level, regulatory frameworks are emerging to guide responsible AI use.

Organizational Oversight is Essential

Companies must define clear internal guidelines for generative AI use:

  • Which tools are authorized?
  • What types of prompts are prohibited?
  • What conversational logs should be retained?

Evolving Regulation

The European AI Act introduces strict controls over high-risk AI systems—especially in critical sectors like cybersecurity, justice, or finance.

Shared Responsibility

AI ethics are not the sole responsibility of the models. They also involve:

  • Developers (responsible design)
  • Vendors (transparency and disclosure)
  • End-users (informed and responsible usage)

CSB.SCHOOL: Preparing Students for the New Frontier of AI and Cybersecurity

At CSB.SCHOOL, we train tomorrow’s cybersecurity specialists through comprehensive programs from post-secondary (Bac) to Bac+5 level, all 100% focused on cybersecurity.

Our programs are co-designed with industry experts and are certified by both ANSSI (through the SecNumEdu label) and the Auvergne-Rhône-Alpes Region, reflecting our deep commitment to academic excellence and rigor.

Today, CSB.SCHOOL is the only school in France to hold both certifications in the cybersecurity field—underscoring our pioneering role in preparing students for the future of AI-augmented cybersecurity.