How must cybersecurity and IAM evolve to mitigate agentic AI risks?

You are currently viewing How must cybersecurity and IAM evolve to mitigate agentic AI risks?
Image generated by Gemini

Introduction: 

 

Agentic AI, which is now being deployed within organisations, represents a new class of AI systems composed of autonomous « agents. » Contrary to generative AI systems that simply generate text or images based on a prompt. Agentic AI is proactive and action-oriented, capable of understanding a task, gathering necessary information, making decisions, and carrying out independent actions, including financial transactions. 

Given their autonomous nature and capability to execute operations, the deployment of Agentic AI raises operational questions concerning cybersecurity.

 

Cyber risks specific to Large Language Models: 

 

Several threats are specific to LLM: 

  • Prompt injection: is an attack vector in which malicious instructions are embedded within the data processed by an agent. Once injected, these instructions can redirect the agent’s actions, leading it to disclose sensitive data or invoke its tools for purposes that serve the attacker rather than the organisation.
  • Memory poisoning: ​​exploits the fact that agents maintain a long-term persistent state to learn and adapt over time. By corrupting this memory, an attacker can alter the agent’s behaviour in a subtle, lasting, and extremely difficult-to-detect manner, thereby turning the agent’s adaptability into a liability.
  • Tool misuse: agents have access to various software tools and APIs to carry out their tasks. A compromised or manipulated agent could abuse these permissions, triggering a cascade of unauthorised actions within an organisation’s digital infrastructure. 

 

Cyber risks specific to Agentic AI systems: 

 

The surface for potential cyberattacks is expanding with AI agents, as they have more access to accomplish their tasks. The use of AI agents increases the risks of data breaches.

  • Privilege risks: these occur because agents are granted permissions at the time of execution, instead of requesting authorisation for each action via traditional identity and access management (IAM) processes. This creates a new category of threat, ‘autonomous privilege escalation’, in which agents accumulate unwanted permissions due to policy deviations, tool chaining or malicious manipulation.   
  • Design and configuration risks: these arise when agents are built or wired with overly broad permissions, static one-time authorisation, and weak segmentation let a single misconfigured or compromised component to reuse stale “allow decisions and move laterally across agents and systems. This extends agents’ affective privilege beyond what was intended. 
  • Behaviour risks: is the situation in which AI agents may act unexpectedly, cause harm, or become exploitable. Adversaries can also steer behaviour via prompt injection, jailbreaks, or poisoned data. 
  • Structural risks: the agentic AI systems are characterised by an interconnected structure between agents, tools, and external data. This interconnection leads to a chain reaction of damage. 

 

A new security paradigm: 

 

The productivity gains of autonomous AI agents capable of performing tasks autonomously come with a proportional increase in the severity of cyberattacks. 

According to McKinsey & Company, the cybersecurity budget in the next three years will rise to 15% due to Agentic AI. The deployment of agent-based AI requires a shift in the approach to cybersecurity. 

Traditionally, cybersecurity has focused on protecting systems operated by humans and enforcing access controls at the perimeter.  Identity verification has become increasingly important with the rise of Software as a Service (SaaS), cloud computing, remote work, and now the emergence of AI agents and other non-human entities. 

Cyber risks associated with AI agents arise in advance of identity authentication. 

With the deployment of AI agents, Identity and Access Management (IAM) must also account for both human and non-human identities to ensure the security of AI agents. 

A security system whose permissions are granted infrequently and reviewed periodically is not adapted to agentic AI. Moreover, current run-time control for agent behaviour lacks reliability. 

 

Ensuring the safety of AI agents: 

 

Security must be embedded into agentic AI from the earliest stages of development. In application of the concept of “privacy by design” to prevent breaches by identifying software and hardware vulnerabilities during the programming phase, and before products reach the market. 

Security by default means AI agents are designed so that the safest, least‑privileged, most privacy‑preserving settings are enabled automatically, without relying on users or operators to harden them manually. Concretely, an agent should fail‑safe (stop and escalate to a human on uncertainty), start with minimal tool/data access, and only expand its permissions or autonomy through explicit, auditable decisions.

Once deployed, agentic systems require continuous monitoring and auditing. For high-impact actions,  humans in the loop must remain mandatory. Organisations must clearly define which actions always require human approval and ensure that agents can’t override these constraints. 

 

Sources: 

Étiquettes: , , ,

Laisser un commentaire

Ce site utilise Akismet pour réduire les indésirables. En savoir plus sur la façon dont les données de vos commentaires sont traitées.