Category: Blog

By
Anahita Bilimoria, Decision Lab Innovation Practice Lead
Sandy Liu, Decision Lab Senior Consultant.

In the past, traditional IT security focused on protecting servers from physical intrusion, malware, and unauthorised network access, sometimes called the fortress model. But in a cloud-native, AI-driven world, threats have evolved. Even if servers remain physically secure, AI models can be manipulated or poisoned remotely, altering outcomes without breaching legacy security practices. Where IT security’s focus was on firewalls, authorisation, and privileges, modern AI security emphasises the integrity of the data and the robustness of the algorithm logics themselves. Because a single malicious input can skew predictions or decision-making, protecting algorithm and its data becomes even more crucial.

Standard software security focuses on patching vulnerabilities, managing identities, and securing APIs. It’s about ensuring the code does only what it’s told to do. If you find a bug, you patch it. If a port is open, you close it. It is deterministic and, for the most part, predictable.

AI flips the script. Unlike traditional software (precise), AI is probabilistic (uncertain). You don’t just secure the code; you have to secure the data its trained on and prompted with, the training process, and the inference logic. AI introduces black-box risks where the system might behave dangerously even if the underlying code is technically bug-free. This is where AI TRiSM (Trust, Risk, and Security Management) becomes essential.

Unlike one-stop shop solutions, AI solutions involve a large group of functionalities interacting with each other. Throughout the solution lifecycle, there are multiple areas that can induce the fear of a functionality being a black box. TRiSM addresses this fear by providing a framework to put multiple layers of security in the solution, ensuring the entirety of the solution follows security measures and builds trust across the solution.  

Within the TRiSM framework, Security Management is the proactive discipline of protecting the entire AI lifecycle. It moves beyond simple IT security to ensure that AI models remain robust, private, and resistant to malicious manipulation.

AI Security vs Traditional Security

Security can’t be an afterthought. Bolting security into solutions after deployment exposes your solution to immense risk. We must adopt a secure-by-design framework in our lifecycle, which starts with identifying and categorising potential threats to the solution.

To understand the changing nature of system security, we can compare traditional software security with AI security across common threats categories, noting that while the categories remain the same, the nature of the risks, and what requires protection, changes.

As this comparison highlights, the attack surface has fundamentally shifted. We are no longer defending against malicious code trying to break into a system; we are guarding against malicious intent attempting to manipulate a model’s logic or corrupt its foundational data. Because the very nature of these threats has evolved, our defensive strategies must evolve in tandem. Let’s break down the specific security measures required to neutralise these new vectors and keep your AI solutions robust.

Type of security measures based on types of attacks

Supply Chain & Data Security

As researchers at the Royal United Services Institute (RUSI) recently highlighted, AI is quietly becoming a major supply chain vulnerability. Attacks targeting this ecosystem focus on compromising training data, external dependencies, or pretrained models used during development. One common example is data poisoning. Another risk involves compromised third-party libraries or pretrained models that may contain hidden vulnerabilities.

Security measures for these attacks focus on ensuring the integrity and trustworthiness of data and external components. Organisations should implement dataset validation processes and maintain clear data provenance records. Dependency scanning tools can help identify vulnerabilities in external libraries, while secure model repositories ensure that only verified artifacts are used during development.

Additional safeguards such as encryption of sensitive datasets, restricted access to training data, and secure data pipelines can further reduce the risk of supply chain attacks affecting the AI system.

Model Integrity

Model integrity is about ensuring the AI remains a faithful, untampered reflection of its intended design. The primary threat here is Data Poisoning (similar to supply chain software solutions), where attackers inject malicious samples into training sets to create backdoors. To counter this, organisations must implement rigorous Data Provenance and Sanitisation protocols, essentially auditing the lineage of every data point to ensure it hasn’t been corrupted.

Adversarial/ Input Security

Even a perfectly trained model can be manipulated once it goes live through Adversarial and Input attacks. The most common threat today is Prompt Injection, where users use jailbreak phrases or clever framing to bypass safety filters. To mitigate this, developers should deploy Prompt Guardrails, which act as a secondary sentinel model that scans incoming requests for malicious intent before they ever reach the primary AI.

In the realm of computer vision or file scanning, attackers often use Adversarial Examples—adding invisible noise to an image or file to cause the AI to misclassify it (e.g., making a stop sign look like a speed limit sign). Building resilience against these tactics requires Adversarial Training, a process where the model is intentionally exposed to broken or attacked samples during development, so it learns to ignore the noise. For high-stakes environments, using Ensemble Methods—where multiple different AI architectures, in effect, vote on a single input—is a highly effective defence, as it is significantly harder for an attacker to fool three different architectures simultaneously than a single, isolated system.

Access Control & API Security

Many AI systems expose their capabilities through APIs, which makes them vulnerable to attacks that attempt to exploit or misuse model access. Security measures in this category focus on controlling and monitoring how users and applications interact with AI models. Strong authentication and authorisation mechanisms should be implemented to ensure that only authorised users can access the system. Role-based access control can limit user permissions based on their responsibilities. Additionally, following industry standards with frameworks like the Model Context Protocol (MCP), allows for a standardised way to manage API calls and link models.

To mitigate automated attacks and excessive queries, organisations should implement rate limiting, request validation, and usage monitoring. Logging and auditing API activity also helps detect abnormal behaviour and potential abuse. By controlling access to AI services, these measures protect the model from exploitation and safeguard sensitive system capabilities.

Deployment & Infrastructure Security

AI models are typically deployed on cloud platforms, containerised environments, or edge infrastructure, which introduces additional attack vectors. Threats in this area may include unauthorised access to the hosting environment, infrastructure misconfigurations, or exploitation of vulnerabilities in the deployment pipeline. Attackers who compromise the infrastructure may gain access to model artifacts, manipulate outputs, or disrupt AI services.

Security measures designed to defend against these attacks focus on protecting the runtime environment and deployment infrastructure. This includes implementing secure configuration practices for cloud resources, isolating AI workloads through containerisation, and encrypting communications between system components.

Integrating security checks into the MLOps or CI/CD pipeline helps identify vulnerabilities before models are deployed. This lifecycle-wide vigilance aligns perfectly with emerging international frameworks like ETSI EN 304 223, which mandates secure practices from initial design right through to operation and retirement. Continuous monitoring of infrastructure activity and system logs can also detect suspicious behavior early. Together, these measures help ensure that AI systems operate within a secure and controlled environment even after deployment.

Apart from forming a security policy, companies must bake them into daily operations of their solution lifecycles. Operationalising security means shifting from reactive patching to proactive, hardened deployment environments. That means stress-testing solutions against both technical failures and adversarial intent.

Conclusion

As we navigate the gold rush of Artificial Intelligence, we must remember a fundamental truth: Unprotected performance isn’t an asset; it’s a liability. A model that is 99% accurate but left vulnerable to data theft or security breaches is not an asset; it is a ticking liability. AI TRiSM allows companies to build a foundation for safe scaling of solutions. Security Management in particular is a pillar that transcends technology types. Whether you are dealing with:

• Hardware (Physical tampering and side-channel attacks),
• Traditional Software (Logic flaws and exploit kits), or
• AI Solutions (Prompt injection and model drift),

The philosophy remains the same. The aim is to introduce security management in every aspect of the solution and not treat it as an afterthought upon deployment. This includes a mindset shift from building a solution to building a secure-by-design solution. We must follow a granular approach and introduce security in the ideation of functionalities, to achieve a robust, anti-fragile and efficient product. By integrating Security management into the solution lifecycle, we help companies ensure trust and dependability.

At Decision Lab, we follow the secure by design approach, so that our solutions excel in current markets, that require anti-fragile, robust solutions. To learn more, contact us!

By Anahita Bilimoria, Decision Lab Innovation Practice Lead

Welcome back to our series on AI TRiSM! In our previous post, we established that Trust is the necessary foundation for AI adoption, built on principles of explainability, fairness, and reliability. However, even the most trusted system carries inherent uncertainties.

The Illusion of Certainty

It is a fundamental fact of data science: while every system is modelled on reality, no model can be a perfect reflection of the real world. Even outside of AI, we accept risk in our most trusted systems:

• Climate Change Models: These are trusted for predicting future warming, yet they involve significant uncertainty (offering a range of possible outcomes) due to the necessary simplification of complex atmospheric, oceanic, and biological interactions.
• Cybersecurity: Highly trusted software systems are constantly patched because determined attackers find zero-day vulnerabilities—flaws the designers didn’t know existed.
• Aviation: While pilots and air traffic control are highly trained, risk is always present due to potential miscommunication or procedural lapses. Checklists and redundancy are built-in specifically to manage this uncertainty.

It is safe to assume that risk is inherently present in all solutions. This brings us to the second, equally crucial pillar of the AI TRiSM framework: Risk Management.

Responsible AI deployment is not about eliminating risk entirely—that is impossible. It is about establishing an effective, proactive strategy for identifying, quantifying, and mitigating it.

In this post, we will:

1. Distinguish between traditional IT risk and unique AI risk.
2. Categorise the specific harm vectors relevant to AI.
3. Outline a four-step framework to operationalise risk management in your organisation.

AI Risk is Not Traditional IT Risk

In traditional IT and cybersecurity, risk is primarily focused on system availability, data security, and compliance breaches. While these still apply, AI introduces unique vectors of harm that require a specialised approach.

The challenge is that AI risks are often non-deterministic—they are tied to the model’s behaviour, not just the infrastructure.

Because these risks move beyond simple system failure, they are trickier to quantify and mitigate.

Categorising AI Risk: The Harm Vectors

To manage AI risk effectively, organisations must classify potential harms into structured categories. These categories provide a blueprint for assessment along with standard mitigation strategies.

1. Performance and Operational Risk

This refers to the risk of the model failing to deliver its intended technical outcomes, or its performance degrading in a real-world environment. This directly impacts Cognitive Trust.

• Model Drift: The model’s real-world data distribution shifts away from the training data, causing accuracy to drop.
  • Mitigation: Implement robust ModelOps monitoring pipelines that continuously compare production performance against established baseline metrics. It is imperative to create pipelines that detect data drift above a certain threshold. If significant drift occurs, the model can be retrained on new data to restore accuracy—frameworks like AgileRL can be instrumental here, offering efficient evolutionary algorithms to accelerate these retraining cycles.
• Adversarial Attacks: Malicious actors introduce subtle, often imperceptible, changes to inputs that trick the model into misclassification (e.g., making a stop sign look like a yield sign to a self-driving car).
  • Mitigation: Employ Adversarial Training during development. Furthermore, organisations can mitigate the risk of non-deterministic AI outputs by pairing them with deterministic Mathematical Optimisation (such as Gurobi). This ensures that even if an AI model acts unpredictably, the final decision is bounded by hard constraints that prevent unsafe or illogical actions.

2. Ethical, Societal, and Reputational Risk

These are risks related to unfairness, bias, lack of transparency, or the unintended negative impact of the AI system on individuals or society. This directly impacts Emotional Trust and brand integrity.

• Bias and Discrimination: The system perpetuates or amplifies historical biases, leading to unfair decisions in high-stakes contexts (e.g., loan applications, hiring, or criminal justice).
  • Mitigation: Conduct Fairness Audits using techniques like disparate impact analysis across protected groups. Implement bias mitigation techniques at every stage of the solution lifecycle. Exploratory Data Analysis (EDA) should be used to highlight data skew that could lead to a biased model.
• Lack of Explainability: The black box nature prevents users or regulators from understanding why a decision was made.
  • Mitigation: Prioritise XAI (Explainable AI) techniques like SHAP and LIME for black-box models, especially in high-consequence decision-making. Where possible, employ inherently white box models (such as Logical Neural Networks) for inbuilt transparency.

3. Security and Compliance Risk

This covers risks related to data privacy, intellectual property theft (model inversion/extraction), and regulatory non-compliance.

• Data Leakage/Privacy Violation: The model inadvertently reveals sensitive training data during inference.
  • Mitigation: Employ Federated Learning (FL), where the model is trained on decentralised edge devices (like smartphones) or local servers. Only model updates (gradients)—not raw data—are sent to the central server. Additionally, Data Sanitisation and Anonymisation ensure that Personal Identifiable Information (PII) is stripped, preventing data from being linked to individuals.
• Regulatory Fines: Failure to adhere to region-specific AI regulations (e.g., the EU AI Act).
  • Mitigation: Establish an AI Governance practice responsible for classifying systems by risk tier. Platforms like Red Hat OpenShift AI can automate this governance, ensuring that mandatory documentation, security protocols, and testing requirements are enforced as a standard part of the solution lifecycle.

Operationalising Risk Management: The Assessment Framework

A responsible organisation integrates AI risk assessment into its existing Enterprise Risk Management (ERM) framework. This process involves four steps:

1. Risk Identification: Map the AI system’s use case to potential harm vectors (e.g., A loan approval model has a high bias risk or A real-time recommendation engine has high model drift risk).
2. Risk Quantification: Estimate the likelihood of the harm occurring and the potential impact (financial, reputational, or societal severity). To do this effectively, organisations can use simulation technology—specifically Digital Twins built with tools like AnyLogic—to test AI models in a risk-free virtual environment before real-world deployment.
3. Risk Mitigation: Implement controls (as listed above) to reduce likelihood and/or impact.
   • Note on Insurance: While software liability is standard, the industry is increasingly discussing AI-specific liability insurance. This emerging sector aims to cover the unique, non-deterministic risks of AI agents that traditional policies might miss.
4. Risk Monitoring: Establish continuous monitoring mechanisms (the Monitoring pillar of TRiSM) to ensure controls remain effective and to catch emerging risks quickly.

The Mandate of Proactive Risk Management

The era of merely deploying a high-performing model and hoping for the best is over. Regulatory bodies across the globe are increasingly making proactive risk assessment a legal mandate.

The AI TRiSM framework provides the discipline to make this transition. It shifts the focus from simply maximising performance metrics to optimising for outcomes across performance, ethics, and security.

By adopting a structured approach to risk, organisations don’t just protect their bottom line—they solidify the trust built with their users and ensure their AI systems are safe, ethical, and sustainable for the long term.

Contact Decision Lab today to learn how our TRiSM-aligned strategies can secure your AI initiatives. Contact us!