AI Era Security Notes
Watch for deepfakes in video calls, prompt injection in AI tools, and AI-generated malware. Combine AI tools with human verification.
Artificial intelligence is a force multiplier for both attackers and defenders. Lesson 7 — the thematic heart of this course — surveys AI-powered threats and practical defensive uses of AI so you can stay ahead in 2026 and beyond.
Prerequisites: Lessons 1–6 recommended. Estimated time: 60–70 minutes.
1. AI on the Offensive
| Threat | Description | Mitigation |
|---|---|---|
| AI-generated phishing | Personalized emails and chats at scale | Out-of-band verification (Lesson 3) |
| Voice cloning / deepfake audio | Fake executive calls authorizing transfers | Callback procedures, code words for finance |
| Deepfake video | Synthetic faces in video meetings | Verify new contacts; use known channels |
| Automated vuln scanning | AI-assisted recon on exposed assets | Attack surface management, patching |
| Malware polymorphism | AI-rewritten payloads evading AV | EDR, behavior-based detection |
| Disinformation | Fake news, synthetic media | Source verification, media literacy |
2. Prompt Injection and LLM Risks
Prompt injection tricks a large language model (LLM) into ignoring instructions — e.g., hidden text in a webpage telling the AI to exfiltrate data. Relevant when organizations embed AI chatbots in products.
Mitigations for builders:
- Separate system prompts from user input strictly
- Output filtering and allowlists for tool/API calls
- Human approval for high-impact actions (payments, deletes)
- Least privilege on tools the AI can invoke
Users: treat AI assistants like untrusted interns — do not give them secrets or unchecked authority.
3. Deepfakes in the Wild
Real incidents include fake video calls where an employee transferred millions after believing a CFO deepfake on a group call. Defenses:
- Establish verification rituals for financial transactions
- Use multi-person approval over threshold amounts
- Train staff that realistic video ≠ authentic person
4. AI on the Defensive
- SIEM/SOAR augmentation — summarize alert floods, suggest playbooks
- UEBA — user entity behavior analytics spotting anomalies
- Phishing detection — ML classifiers on email gateways
- Threat intelligence — faster correlation of IOCs
- Secure code assistants — with human review, catch common bugs
AI assists analysts; it does not replace accountability. Always validate high-stakes AI output.
5. AI Governance and Policy
Organizations should define:
- Approved AI tools and data classification rules
- Prohibition on pasting customer PII into public models
- Incident playbooks for AI misuse or model leaks
- Vendor due diligence (training data, retention, compliance)
6. What Is Coming Next
Expect agentic AI (autonomous multi-step tools), more regulation (EU AI Act influence globally), and arms races in deepfake detection. Continuous learning is part of cybersecurity in the AI era.
7. Scenario: AI Support Scam
A pop-up says your PC is infected and offers a “Microsoft AI Support” chat. The bot sounds helpful and asks you to install remote access software.
Response: Close the browser. Never grant remote access to unsolicited callers or chats. Real vendors do not operate this way. Run a scan with your installed EDR if you downloaded anything.
8. Next Steps
Continue to Lesson 8 — Real-World Defense and Incident Response to tie everything together.
Frequently Asked Questions
What is prompt injection?
An attack where malicious input causes an AI system to ignore its rules or leak data — critical for embedded chatbots and agents.
Can AI help defenders too?
Yes. Security teams use AI for anomaly detection, log analysis, and faster triage — combined with human judgment.
How do I spot a deepfake video call?
Look for unnatural blinking, lighting inconsistencies, and audio lag — but process beats pixels: verify through a separate trusted channel.
Want live cybersecurity classes? Join Alkademy for instructor-led cybersecurity courses with hands-on projects.