Prompt Injection Is the New Phishing: Defending AI Tools From Manipulated Inputs

Phishing works by tricking a person into following malicious instructions disguised as legitimate ones. Prompt injection does the same thing to AI. And as organizations give AI assistants and agents access to email, documents, browsers and business systems, it is quickly becoming one of the most important security risks in the enterprise.

How prompt injection works

Large language models take instructions in plain language. The problem is that they can struggle to tell the difference between instructions from their legitimate user and instructions hidden in the content they process.

Direct prompt injection happens when a user deliberately types instructions designed to override an AI system’s rules, for example to extract confidential system prompts or bypass safety controls.

Indirect prompt injection is the bigger enterprise threat. Here, malicious instructions are hidden inside content the AI reads on a user’s behalf: a web page, an email, a PDF, a calendar invite, a support ticket or a code repository. The instructions might be invisible to humans, written in white text, tucked into metadata or buried in a long document.

Imagine an AI assistant asked to summarize a user’s inbox. One email contains hidden text instructing the AI to search for password reset messages and forward them to an external address. If the assistant has permission to send email and lacks adequate safeguards, it may comply. The user never clicked anything.

Why it’s like phishing, only harder

Like phishing, prompt injection exploits trust and the difficulty of distinguishing legitimate instructions from malicious ones. But AI agents can act at machine speed, across many systems, with whatever permissions they’ve been given. And unlike humans, they don’t get suspicious when something feels off unless they’ve been designed to.

What’s at risk

• Data exfiltration through emails, API calls or links that embed sensitive data
• Unauthorized actions such as sending messages, changing records, approving transactions or modifying code
• Manipulated outputs that mislead users, such as altered summaries, poisoned search results or biased recommendations
• Credential and system exposure when agents can access secrets, tools or internal systems

Defending your AI tools

No single control eliminates prompt injection today, so defense in depth is essential.

1. Limit agent permissions. Give AI tools only the access they absolutely need. An assistant that summarizes documents shouldn’t be able to send email or delete files.

2. Require human approval for sensitive actions. Sending external messages, moving money, changing permissions or deleting data should require explicit user confirmation.

3. Treat external content as untrusted. Design systems so content from emails, web pages and documents is handled as data, not instructions, and clearly separated from trusted system prompts.

4. Restrict outbound channels. Limit where agents can send data, such as blocking arbitrary external URLs or unapproved email recipients.

5. Monitor and log agent activity. Record what AI agents read, decide and do so unusual behavior can be detected and investigated.

6. Test adversarially. Red-team your AI applications with prompt injection scenarios before and after deployment, and evaluate vendors on how they defend against these attacks.

7. Educate users. Help employees understand that AI tools can be manipulated by content and should not be granted broad access casually.

The bottom line

As AI moves from answering questions to taking actions, manipulated inputs become a direct path to real-world harm. Security teams that apply least privilege, human approval, monitoring and adversarial testing to AI agents will be far better prepared for the attack technique that’s poised to define the next era of phishing.