Tag: AI TRiSM

  • Understanding AI Security: Why Traditional IT Defences Are No Longer Enough

    Understanding AI Security: Why Traditional IT Defences Are No Longer Enough

    By
    Anahita Bilimoria, Decision Lab Innovation Practice Lead
    Sandy Liu, Decision Lab Senior Consultant.

    In the past, traditional IT security focused on protecting servers from physical intrusion, malware, and unauthorised network access, sometimes called the fortress model. But in a cloud-native, AI-driven world, threats have evolved. Even if servers remain physically secure, AI models can be manipulated or poisoned remotely, altering outcomes without breaching legacy security practices. Where IT security’s focus was on firewalls, authorisation, and privileges, modern AI security emphasises the integrity of the data and the robustness of the algorithm logics themselves. Because a single malicious input can skew predictions or decision-making, protecting algorithm and its data becomes even more crucial.

    Standard software security focuses on patching vulnerabilities, managing identities, and securing APIs. It’s about ensuring the code does only what it’s told to do. If you find a bug, you patch it. If a port is open, you close it. It is deterministic and, for the most part, predictable.

    AI flips the script. Unlike traditional software (precise), AI is probabilistic (uncertain). You don’t just secure the code; you have to secure the data its trained on and prompted with, the training process, and the inference logic. AI introduces black-box risks where the system might behave dangerously even if the underlying code is technically bug-free. This is where AI TRiSM (Trust, Risk, and Security Management) becomes essential.

    Unlike one-stop shop solutions, AI solutions involve a large group of functionalities interacting with each other. Throughout the solution lifecycle, there are multiple areas that can induce the fear of a functionality being a black box. TRiSM addresses this fear by providing a framework to put multiple layers of security in the solution, ensuring the entirety of the solution follows security measures and builds trust across the solution.  

    Within the TRiSM framework, Security Management is the proactive discipline of protecting the entire AI lifecycle. It moves beyond simple IT security to ensure that AI models remain robust, private, and resistant to malicious manipulation.

    AI Security vs Traditional Security

    Security can’t be an afterthought. Bolting security into solutions after deployment exposes your solution to immense risk. We must adopt a secure-by-design framework in our lifecycle, which starts with identifying and categorising potential threats to the solution.

    To understand the changing nature of system security, we can compare traditional software security with AI security across common threats categories, noting that while the categories remain the same, the nature of the risks, and what requires protection, changes.

    Threat CategoryTraditional SecurityAI Security
    Social/InputPhishing: Tricking a user into giving up a password.Prompt Injection: Tricking a model into ignoring its guardrails to leak data or execute commands.
    InfectionMalware: Malicious code designed to corrupt a system.Adversarial Attacks: Subtly altered inputs (like invisible noise on an image) that cause a model to malfunction.
    Service DisruptDDoS: Flooding a server with traffic to take it offline.Model Inversion / Drift: Stealing the model’s logic via queries, or the model becoming stale and inaccurate over time.
    Data IntegrityMan-in-the-Middle: Intercepting data as it moves between points.Data Poisoning: Contaminating the training data so the model learns a backdoor or bias.

    As this comparison highlights, the attack surface has fundamentally shifted. We are no longer defending against malicious code trying to break into a system; we are guarding against malicious intent attempting to manipulate a model’s logic or corrupt its foundational data. Because the very nature of these threats has evolved, our defensive strategies must evolve in tandem. Let’s break down the specific security measures required to neutralise these new vectors and keep your AI solutions robust.

    Type of security measures based on types of attacks

    An isometric diagram titled "AI Security Ecosystem" illustrating five key security pillars connected to a central "Secure AI Framework" node. The five pillars are: Supply Chain & Data Security (secure dependencies and data provenance); Model Integrity (rigorous sanitisation protocols); Adversarial/Input Security (prompts guardrails and adversarial training); Access Control & API Security (authorisation and usage monitoring); and Deployment & Infrastructure Security (proactive hardened environments).

    Supply Chain & Data Security

    As researchers at the Royal United Services Institute (RUSI) recently highlighted, AI is quietly becoming a major supply chain vulnerability. Attacks targeting this ecosystem focus on compromising training data, external dependencies, or pretrained models used during development. One common example is data poisoning. Another risk involves compromised third-party libraries or pretrained models that may contain hidden vulnerabilities.

    Security measures for these attacks focus on ensuring the integrity and trustworthiness of data and external components. Organisations should implement dataset validation processes and maintain clear data provenance records. Dependency scanning tools can help identify vulnerabilities in external libraries, while secure model repositories ensure that only verified artifacts are used during development.

    Additional safeguards such as encryption of sensitive datasets, restricted access to training data, and secure data pipelines can further reduce the risk of supply chain attacks affecting the AI system.

    Model Integrity

    Model integrity is about ensuring the AI remains a faithful, untampered reflection of its intended design. The primary threat here is Data Poisoning (similar to supply chain software solutions), where attackers inject malicious samples into training sets to create backdoors. To counter this, organisations must implement rigorous Data Provenance and Sanitisation protocols, essentially auditing the lineage of every data point to ensure it hasn’t been corrupted.

    Adversarial/ Input Security

    Even a perfectly trained model can be manipulated once it goes live through Adversarial and Input attacks. The most common threat today is Prompt Injection, where users use jailbreak phrases or clever framing to bypass safety filters. To mitigate this, developers should deploy Prompt Guardrails, which act as a secondary sentinel model that scans incoming requests for malicious intent before they ever reach the primary AI.

    In the realm of computer vision or file scanning, attackers often use Adversarial Examples—adding invisible noise to an image or file to cause the AI to misclassify it (e.g., making a stop sign look like a speed limit sign). Building resilience against these tactics requires Adversarial Training, a process where the model is intentionally exposed to broken or attacked samples during development, so it learns to ignore the noise. For high-stakes environments, using Ensemble Methods—where multiple different AI architectures, in effect, vote on a single input—is a highly effective defence, as it is significantly harder for an attacker to fool three different architectures simultaneously than a single, isolated system.

    Access Control & API Security

    Many AI systems expose their capabilities through APIs, which makes them vulnerable to attacks that attempt to exploit or misuse model access. Security measures in this category focus on controlling and monitoring how users and applications interact with AI models. Strong authentication and authorisation mechanisms should be implemented to ensure that only authorised users can access the system. Role-based access control can limit user permissions based on their responsibilities. Additionally, following industry standards with frameworks like the Model Context Protocol (MCP), allows for a standardised way to manage API calls and link models.

    To mitigate automated attacks and excessive queries, organisations should implement rate limiting, request validation, and usage monitoring. Logging and auditing API activity also helps detect abnormal behaviour and potential abuse. By controlling access to AI services, these measures protect the model from exploitation and safeguard sensitive system capabilities.

    Deployment & Infrastructure Security

    AI models are typically deployed on cloud platforms, containerised environments, or edge infrastructure, which introduces additional attack vectors. Threats in this area may include unauthorised access to the hosting environment, infrastructure misconfigurations, or exploitation of vulnerabilities in the deployment pipeline. Attackers who compromise the infrastructure may gain access to model artifacts, manipulate outputs, or disrupt AI services.

    Security measures designed to defend against these attacks focus on protecting the runtime environment and deployment infrastructure. This includes implementing secure configuration practices for cloud resources, isolating AI workloads through containerisation, and encrypting communications between system components.

    Integrating security checks into the MLOps or CI/CD pipeline helps identify vulnerabilities before models are deployed. This lifecycle-wide vigilance aligns perfectly with emerging international frameworks like ETSI EN 304 223, which mandates secure practices from initial design right through to operation and retirement. Continuous monitoring of infrastructure activity and system logs can also detect suspicious behavior early. Together, these measures help ensure that AI systems operate within a secure and controlled environment even after deployment.

    Apart from forming a security policy, companies must bake them into daily operations of their solution lifecycles. Operationalising security means shifting from reactive patching to proactive, hardened deployment environments. That means stress-testing solutions against both technical failures and adversarial intent.

    Conclusion

    As we navigate the gold rush of Artificial Intelligence, we must remember a fundamental truth: Unprotected performance isn’t an asset; it’s a liability. A model that is 99% accurate but left vulnerable to data theft or security breaches is not an asset; it is a ticking liability. AI TRiSM allows companies to build a foundation for safe scaling of solutions. Security Management in particular is a pillar that transcends technology types. Whether you are dealing with:

    • Hardware (Physical tampering and side-channel attacks),
    • Traditional Software (Logic flaws and exploit kits), or
    • AI Solutions (Prompt injection and model drift),

    The philosophy remains the same. The aim is to introduce security management in every aspect of the solution and not treat it as an afterthought upon deployment. This includes a mindset shift from building a solution to building a secure-by-design solution. We must follow a granular approach and introduce security in the ideation of functionalities, to achieve a robust, anti-fragile and efficient product. By integrating Security management into the solution lifecycle, we help companies ensure trust and dependability.

    At Decision Lab, we follow the secure by design approach, so that our solutions excel in current markets, that require anti-fragile, robust solutions. To learn more, contact us!

  • Why LLMs Aren’t Enough: Engineering Antifragile Operations with Composable Decision Intelligence

    Why LLMs Aren’t Enough: Engineering Antifragile Operations with Composable Decision Intelligence

    As we navigate the technological landscape of 2026, Generative AI has undoubtedly transformed the way we interact with information. Chatbots and Large Language Models (LLMs) have proliferated across enterprise software, streamlining communication and automating basic workflows. However, for operations and supply chain leaders in complex, capital-intensive industries like FMCG, Automotive, and Retail manufacturing, a stark reality is emerging: LLMs are not a silver bullet.

    While language models excel at processing text, they cannot single-handedly optimise a global supply chain network, nor can they provide the quantitative assurance needed to de-risk a £50m factory expansion. When dealing with physical realities, extreme market volatility, and fragmented legacy systems, text prediction is insufficient.

    By 2026, 75% of Global 500 companies will apply decision intelligence practices

    Gartner

    The definitive competitive edge in 2026 belongs to those looking beyond Generative AI toward Decision AI. It belongs to organisations thoughtfully advancing their tech stack and building on established capabilities to embrace the architecture of a Composable Decision Intelligence Platform (DIP).

    The Industrial Reality: High Stakes and High Volatility

    Traditional planning systems struggle to account for agile and accelerated business and the consequent hazards. Today’s supply chain and operations leaders are caught in a crossfire of overlapping challenges, two of the most critical being:

    • Demand & Supply Volatility: SKU proliferation, shifting consumer behaviours, and frequent supply chain disruptions are breaking static planning models. The inability of legacy systems to cope with this extreme volatility inevitably results in poor service levels, excess inventory, and spiralling costs.
    • High-Stakes CAPEX Uncertainty: Securing funding for major capital investments—whether a new automated line, a facility expansion, or rationalising a post-merger manufacturing network—requires robust, data-driven justification. Without quantitative assurance, it is incredibly difficult to de-risk these investments and guarantee ROI.

    Solving these multi-dimensional problems requires more than just analysing past data; it requires a platform capable of simulating the future and discovering the optimal path forward.

    The Path to Implementation: A Composable Architecture

    At Decision Lab, we deliver Decision Intelligence to help leaders master this uncertainty. We achieve this not through a rigid, black-box AI model, but by building a Composable Decision Intelligence Platform based on responsible AI TRiSM principles.

    Composability is the principle that enables businesses to be agile. Rather than relying on a single vendor’s inflexible suite, a composable DIP orchestrates best-in-class, modular capabilities that ingest data from fragmented ERP, MES, and WMS systems. This creates a unified, dynamic view—an AI Simulation Twin.

    The Strategic Advantage of the AI Simulation Twin

    Instead of waiting years for a fully instrumented, hardware-dependent Digital Twin, leading organisations are accelerating their time-to-value by deploying an AI Simulation Twin.

    Traditional Digital Twin programmes often stall in pilot purgatory due to immense IoT integration challenges, prohibitive hardware costs, and fragmented legacy data pipelines. A Simulation Twin, while still ingesting real data, fundamentally bypasses these immediate infrastructure hurdles. It delivers the core predictive and prescriptive advantages now—providing a high-fidelity virtual environment to solve urgent CAPEX and operational bottlenecks—while your physical IoT maturity can be developed as a separate, parallel track. This decoupling ensures you realise ROI in months, rather than years, before moving into the four pillars of the platform:

    Infographic of the four pillars of a composable decision intelligence platform.

    1. The Cognitive Engine: Autonomous AI Agents

    Agentic AI serves as the reasoning layer of the platform. These agents can interpret complex scenarios, model market volatility, and process multi-tiered supply chain dynamics, translating raw data into actionable context.

    2. The Virtual Sandbox: Simulation

    To understand a complex physical network, you must be able to interrogate it and test it. Practically, that means replicating it. We use simulation to build a high-fidelity digital twin environment, employing appropriate technologies, such as AnyLogic’s multi-method capabilities. A simulation maps constraints, machines, and distribution nodes, providing the holistic view necessary to test what-if scenarios safely. It answers critical CAPEX questions before money is spent.

    3. The Mathematical Engine: Optimisation

    Where simulation shows you what could happen, optimisation dictates what should happen. For us, that means employing mathematical optimisation, such as Gurobi’s world-class mathematical solver, to cut through millions of potential permutations. It discovers the mathematically perfect production schedules and inventory policies—maximising throughput and service levels while minimising duplicated costs. The key is being timely—it is no good getting the answer after it was needed. Gurobi’s speed is key here (Gurobi white paper on solver speed).

    4. The Continuous Learning Loop: Reinforcement Learning

    This is where the platform moves from a passive analytical tool to an active operational asset. By applying Reinforcement Learning, specifically leveraging AgileRL, a platform can learn from real-time feedback. It continually experiments within the simulation, discovering new strategies to navigate supply shocks or demand spikes as they happen.

    Engineering the Antifragile Supply Chain

    The ultimate goal of implementing a Composable Decision Intelligence Platform is to shift operations from a state of fragility to one of Antifragility.

    A robust system merely survives a shock. An antifragile operational system improves when exposed to volatility. When a sudden supply chain disruption occurs, the reinforcement learning algorithms immediately assess the new reality within the simulation, trigger the optimisation engine to recalculate the best path, and deploy autonomous agents to orchestrate a self-adapting response. Relying on singular AI models or monolithic ERPs to solve complex physical problems is being consigned to the past.

    For leaders navigating constant disruption, true agility requires an adaptable, composable ecosystem. By implementing a Decision Intelligence Platform, you gain the foresight not just to predict the future, but to engineer your position—a compelling competitive advantage now and for the future.

    To find out more, check out our case studies or contact us!

  • AI TRiSM: Moving From Theory to Action with Risk Management

    AI TRiSM: Moving From Theory to Action with Risk Management

    By Anahita Bilimoria, Decision Lab Innovation Practice Lead

    Welcome back to our series on AI TRiSM! In our previous post, we established that Trust is the necessary foundation for AI adoption, built on principles of explainability, fairness, and reliability. However, even the most trusted system carries inherent uncertainties.

    The Illusion of Certainty

    It is a fundamental fact of data science: while every system is modelled on reality, no model can be a perfect reflection of the real world. Even outside of AI, we accept risk in our most trusted systems:

    • Climate Change Models: These are trusted for predicting future warming, yet they involve significant uncertainty (offering a range of possible outcomes) due to the necessary simplification of complex atmospheric, oceanic, and biological interactions.
    • Cybersecurity: Highly trusted software systems are constantly patched because determined attackers find zero-day vulnerabilities—flaws the designers didn’t know existed.
    • Aviation: While pilots and air traffic control are highly trained, risk is always present due to potential miscommunication or procedural lapses. Checklists and redundancy are built-in specifically to manage this uncertainty.

    It is safe to assume that risk is inherently present in all solutions. This brings us to the second, equally crucial pillar of the AI TRiSM framework: Risk Management.

    Responsible AI deployment is not about eliminating risk entirely—that is impossible. It is about establishing an effective, proactive strategy for identifying, quantifying, and mitigating it.

    In this post, we will:

    1. Distinguish between traditional IT risk and unique AI risk.
    2. Categorise the specific harm vectors relevant to AI.
    3. Outline a four-step framework to operationalise risk management in your organisation.

    AI Risk is Not Traditional IT Risk

    In traditional IT and cybersecurity, risk is primarily focused on system availability, data security, and compliance breaches. While these still apply, AI introduces unique vectors of harm that require a specialised approach.

    The challenge is that AI risks are often non-deterministic—they are tied to the model’s behaviour, not just the infrastructure.

    Traditional IT RiskUnique AI Risk
    System Outage (Downtime)Model Drift (Degradation of accuracy over time)
    Data Breach (Unauthorised access)Data Poisoning (Malicious manipulation of training data)
    Compliance Failure (e.g., missed deadlines)Algorithmic Bias (Discriminatory outcomes)
    Software Vulnerability (e.g., zero-day exploit)Model Hallucination (Generating false but plausible outputs)

    Because these risks move beyond simple system failure, they are trickier to quantify and mitigate.

    Categorising AI Risk: The Harm Vectors

    To manage AI risk effectively, organisations must classify potential harms into structured categories. These categories provide a blueprint for assessment along with standard mitigation strategies.

    1. Performance and Operational Risk

    This refers to the risk of the model failing to deliver its intended technical outcomes, or its performance degrading in a real-world environment. This directly impacts Cognitive Trust.

    • Model Drift: The model’s real-world data distribution shifts away from the training data, causing accuracy to drop.
      • Mitigation: Implement robust ModelOps monitoring pipelines that continuously compare production performance against established baseline metrics. It is imperative to create pipelines that detect data drift above a certain threshold. If significant drift occurs, the model can be retrained on new data to restore accuracy—frameworks like AgileRL can be instrumental here, offering efficient evolutionary algorithms to accelerate these retraining cycles.
    • Adversarial Attacks: Malicious actors introduce subtle, often imperceptible, changes to inputs that trick the model into misclassification (e.g., making a stop sign look like a yield sign to a self-driving car).
      • Mitigation: Employ Adversarial Training during development. Furthermore, organisations can mitigate the risk of non-deterministic AI outputs by pairing them with deterministic Mathematical Optimisation (such as Gurobi). This ensures that even if an AI model acts unpredictably, the final decision is bounded by hard constraints that prevent unsafe or illogical actions.

    2. Ethical, Societal, and Reputational Risk

    These are risks related to unfairness, bias, lack of transparency, or the unintended negative impact of the AI system on individuals or society. This directly impacts Emotional Trust and brand integrity.

    • Bias and Discrimination: The system perpetuates or amplifies historical biases, leading to unfair decisions in high-stakes contexts (e.g., loan applications, hiring, or criminal justice).
      • Mitigation: Conduct Fairness Audits using techniques like disparate impact analysis across protected groups. Implement bias mitigation techniques at every stage of the solution lifecycle. Exploratory Data Analysis (EDA) should be used to highlight data skew that could lead to a biased model.
    • Lack of Explainability: The black box nature prevents users or regulators from understanding why a decision was made.
      • Mitigation: Prioritise XAI (Explainable AI) techniques like SHAP and LIME for black-box models, especially in high-consequence decision-making. Where possible, employ inherently white box models (such as Logical Neural Networks) for inbuilt transparency.

    3. Security and Compliance Risk

    This covers risks related to data privacy, intellectual property theft (model inversion/extraction), and regulatory non-compliance.

    • Data Leakage/Privacy Violation: The model inadvertently reveals sensitive training data during inference.
      • Mitigation: Employ Federated Learning (FL), where the model is trained on decentralised edge devices (like smartphones) or local servers. Only model updates (gradients)—not raw data—are sent to the central server. Additionally, Data Sanitisation and Anonymisation ensure that Personal Identifiable Information (PII) is stripped, preventing data from being linked to individuals.
    • Regulatory Fines: Failure to adhere to region-specific AI regulations (e.g., the EU AI Act).
      • Mitigation: Establish an AI Governance practice responsible for classifying systems by risk tier. Platforms like Red Hat OpenShift AI can automate this governance, ensuring that mandatory documentation, security protocols, and testing requirements are enforced as a standard part of the solution lifecycle.

    Operationalising Risk Management: The Assessment Framework

    A responsible organisation integrates AI risk assessment into its existing Enterprise Risk Management (ERM) framework. This process involves four steps:

    1. Risk Identification: Map the AI system’s use case to potential harm vectors (e.g., A loan approval model has a high bias risk or A real-time recommendation engine has high model drift risk).
    2. Risk Quantification: Estimate the likelihood of the harm occurring and the potential impact (financial, reputational, or societal severity). To do this effectively, organisations can use simulation technology—specifically Digital Twins built with tools like AnyLogic—to test AI models in a risk-free virtual environment before real-world deployment.
    3. Risk Mitigation: Implement controls (as listed above) to reduce likelihood and/or impact.
      • Note on Insurance: While software liability is standard, the industry is increasingly discussing AI-specific liability insurance. This emerging sector aims to cover the unique, non-deterministic risks of AI agents that traditional policies might miss.
    4. Risk Monitoring: Establish continuous monitoring mechanisms (the Monitoring pillar of TRiSM) to ensure controls remain effective and to catch emerging risks quickly.

    The Mandate of Proactive Risk Management

    The era of merely deploying a high-performing model and hoping for the best is over. Regulatory bodies across the globe are increasingly making proactive risk assessment a legal mandate.

    The AI TRiSM framework provides the discipline to make this transition. It shifts the focus from simply maximising performance metrics to optimising for outcomes across performance, ethics, and security.

    By adopting a structured approach to risk, organisations don’t just protect their bottom line—they solidify the trust built with their users and ensure their AI systems are safe, ethical, and sustainable for the long term.

    Contact Decision Lab today to learn how our TRiSM-aligned strategies can secure your AI initiatives. Contact us!

  • Trust in AI systems

    Trust in AI systems

    By Anahita Bilimoria, Decision Lab Innovation Practice Lead

    An essential framework for Responsible AI Deployment

    In this blog post, we dive deeper into the AI TRiSM principle of Trust. Together, the principles of AI TRiSM (Trust, Risk, and Security Management) add transparency, understandability, and reliability to our AI systems.

    Continuing from our previous blog on AI TRiSM, Building Trust in the Age of Artificial Intelligence, where we took a holistic view of the three core pillars of AI TRiSM, this blog post dives deeper into the principle of Trust. Welcome to part two of our AI TRiSM series: Trust in AI systems.

    The Foundation of Adoption: Building Trust in AI

    AI has integrated into our daily lives in unprecedented ways, from using Gemini or ChatGPT to summarise reports to utilising tools like Google’s NotebookLM for learning. However, the reliability of answers given by AI systems often remains uncertain. While we confidently ask a Language Model for code, broad trust in AI systems is still a major concern.

    In the context of AI, a more fitting definition may be: the attitude that an AI agent will help achieve an individual’s goals in a situation characterised by uncertainty and vulnerability. Clearly, building trust goes beyond simple belief in their capabilities—AI chatbots can seem remarkably confident. Rather, we must establish a calibrated trust: confidence that an AI system will behave reliably, ethically, and securely, leading to intended outcomes while understanding its limitations. For the human in the system, this confidence operates on two crucial dimensions:

    • Cognitive Trust: based on evidence, competence, and reliability. Confirmed by performance metrics such as accuracy, loss, and F1 score, ensuring the system works as expected. Answers the question: can I trust it?
    • Emotional Trust: based on comfort, security, and ethical alignment. The confidence that the system aligns with moral or societal values and will not discriminate against users. Answers the question: do I want to trust it?

    The goal of AI TRiSM is to foster a balanced level of trust across these dimensions, steering clear of algorithm aversion (distrusting a competent system) and automation complacency (over-relying on a flawed system). Trust in AI is not a single feature but the culmination of several measurable and governable qualities. The AI TRiSM framework tackles these factors by providing actionable strategies:

    1. Explainable and Transparent (XAI)

    The defining challenge for trust is the ‘Black Box’ nature of modern, complex AI models. If a system’s decision cannot be understood or audited, it fundamentally cannot be trusted.

    Explainable AI (XAI) addresses this by providing insight into how and why a model reached a specific output. This is essential not just for a user’s peace of mind, but for auditing, compliance, and legal accountability.

    How it can be achieved:

    • Employing Inherently Interpretable Models: Using simpler models (like linear regression or decision trees) when complexity isn’t strictly necessary. Utilising inherent white-box models such as NeSy models (Neurosymbolic AI) and Causal ML routes that provide complete knowledge graphs of model knowledge.
    • Providing Decision Tracing: Logging all input features and intermediate steps that lead to an outcome, allowing stakeholders to trace a decision back to its source. Applying methods like SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model-agnostic Explanations) to complex models to generate human-readable justifications for individual predictions.

    2. Fairness and Bias Mitigation

    AI models are trained on data, and that data reflects historical and societal biases. Consequently, models are prone to inheriting these biases, leading to discriminatory or unfair outcomes. This directly breaks emotional trust. Building trust requires active and continuous steps to ensure fairness.

    How it can be achieved:

    • Pre-Processing Bias Mitigation: Conducting thorough Exploratory Data Analysis (EDA) to identify and balance data imbalances before model training (e.g., re-sampling minority classes).
    • Model Bias Mitigation: Implementing constraints during the training process that penalize the model for differential performance across different demographic groups. Defining and monitoring multiple fairness metrics (like equal opportunity or demographic parity) after deployment to ensure equal outcomes across protected groups, going beyond simple overall accuracy.

    3. Reliability, Robustness, and Safety

    A trusted system must be dependable. Reliability is its ability to perform consistently and accurately under normal operating conditions. Robustness ensures the model’s accuracy is maintained even when facing slight variations or unexpected inputs. The final layer is Safety, which protects against catastrophic failure.

    How it can be achieved:

    • Continuous Model Operations (ModelOps): Implementing automated systems to monitor model performance in real-time, catching model drift (when performance degrades over time) or degradation after deployment.
    • Stress Testing and Adversarial Training: Rigorously testing the model with malicious inputs and unexpected data shifts to improve robustness against adversarial attacks.
    • Human-in-the-Loop Controls: Equipped with safeguards like “kill switches” and defined pathways for human intervention to ensure an autonomous system can be overridden or stopped when faced with an unsafe or ambiguous situation.

    4. Privacy and Data Protection

    In the age of vast data collection, a user’s willingness to use an AI solution hinges on the assurance that their sensitive information will be protected. Trust is lost if data is compromised, misused, or leaked. Adhering to regulations like GDPR and CCPA is a baseline.

    How it can be achieved:

    • Secure-by-Design Principles: Initiating and maintaining AI solution development that incorporates techniques such as anonymisation and data minimisation (minimum necessary data collection).
    • Privacy-Enhancing Technologies (PETs): Utilising advanced cryptographic techniques like federated learning (training models on decentralised data) to protect sensitive information during training and inference.
    • Access Control and Security Audits: Implementing strict access controls and regular security audits for data pipelines and model APIs to ensure compliance and prevent unauthorised data access.

    The Value of Proactive Trust Management

    The journey of AI adoption is paved with the potential for misuse and technical failure. A single, high-profile failure, such as a biased recommendation, a security breach, or a dangerous hallucination, can instantly erode years of trust-building effort.

    By embracing the Trust component of AI TRiSM, organisations can move from reactive damage control to proactive trust management. They can operationalise these ethical and performance requirements, embedding them into the entire solution lifecycle.

    Investing in these principles is an investment in the long-term viability of AI, ensuring that as systems become more autonomous and integrated into our lives, they remain aligned with our values, transparent in their operations, and secure with our data. This is how we build systems that don’t just perform a task reliably, but adapt and improve in a volatile world. This concept is particularly critical in complex, high-stakes environments like supply chain management, where disruption is a constant threat.

    For a deeper exploration of how these principles are applied to create systems that gain from disorder, see our recent white paper: Beyond Resilience: Engineering the Anti-Fragile Pharma Supply Chain of 2030.

    The next in the AI TRiSM series: Moving From Theory to Action with Risk Management

    Author: Anahita Bilimoria, Decision Lab Innovation Practice Lead.
    Follow this series on AI TRiSM from Decision Lab, follow us on LinkedIn!

  • AI TRiSM: Building Trust in the Age of Artificial Intelligence

    AI TRiSM: Building Trust in the Age of Artificial Intelligence

    By Anahita Bilimoria, Decision Lab Senior Machine-Learning Engineer

    An Essential Framework for Responsible AI Deployment

    The promise of Artificial Intelligence is immense, offering solutions to humanity’s most pressing challenges. With the recent boom of Language Models and AI penetrating every domain, we are confronted with a fundamental truth: global adoption and subsequent progress rely solely on trust. While the performance of AI in every aspect of automated decision-making is phenomenal, there has been a rising concern for trust in AI, fueled by opaque decision-making and perceived biases. This challenge has given rise to stalled innovation, public apprehension and the risk of deploying technologies without adequate oversight. The urgency to build and maintain trust in AI has surpassed being simply a matter of ethics and has given rise to safety concerns as well.

    Despite adaptive speed and response, autonomous AI’s unchecked deployment can lead to instability. The absence of clear dependability metrics and inadequate interpretability methods raise trust questions across diverse AI. Although some transparent AI exists, rapid critical adoption in a volatile environment, coupled with regulation and public concern, necessitates urgent trust-building. AI TRiSM provides a repeatable framework (trust, security, privacy, transparency) to address these risks.

    What is AI TRiSM?

    Gartner, a leading research and advisory company, defines AI TRiSM (AI Trust, Risk and Security Management) as a framework that ‘ensures AI model governance, trustworthiness, fairness, reliability, robustness, efficacy, and data protection’. This ensures that models that follow this framework are not unethical, unfair or biased. While most AI solutions focus on model performance, AI TRiSM adds a layer of model responsibility, urging developers to strike a balance between the two.

    A hexagonal diagram illustrating key metrics for AI Model Performance, a core component of AI-TRiSM. Metrics shown include accuracy, F1 score, loss, and precision.
    A diagram of AI Model Responsibility's core components. A central hexagon is surrounded by six related principles: Transparency, Explainability, Fairness, Bias, Security, and Privacy.

    Figure 1: Balancing Model Performance and Model Responsibility. This diagram illustrates the key metrics associated with traditional AI Model Performance (such as Accuracy, F1 score, and Loss) in contrast with the crucial metrics for Model Responsibility (including Transparency, Explainability, Fairness, and Security) that are central to AI TRiSM.

    Despite being a framework of individual principles, AI TRiSM enables the fulfillment of each principle through conscious and targeted steps.

    Trust

    Entailing the concepts of transparency, fairness, reliability, privacy and safety; this pillar ensures that models offer accountability and build trust. This component of AI TRiSM requires models to offer explainability, either by using Explainable AI (xAI), which are models designed to be interpretable, or artificially inducing explainability in their decision-making processes. Models trained on data are prone to biases in the data itself, leading to discriminatory model outcomes. Techniques like a thorough Exploratory Data Analysis (EDA) – which involves visualizing and summarizing data to spot imbalances – and bias detection methods can help gain insight into the biases in data. Regulations like GDPR and CCPA ensure data privacy and security. While models can’t follow these regulations directly, you can ensure your AI solution does by implementing appropriate data handling and storage practices! Autonomous solutions can build trust by introducing kill switches, pathways for human intervention, and safe operation mechanisms to ensure safe execution even in unexpected situations.

    Risk

    This pillar involves identifying and managing risks associated with your AI solution throughout its lifecycle. You can map up the risks associated with your solution. Some key aspects can include:

    • Performance risks (e.g., model drift, accuracy degradation)
    • Ethical risks (e.g., bias, lack of fairness)
    • Security risks (e.g., adversarial attacks, data breaches)
    • Operational risks (e.g., deployment failures, integration issues)

    The solution lifecycle must include allocated resources for identifying, evaluating, and mitigating risks proactively throughout the solution’s development and deployment. Identifying these risks proactively helps stakeholders make informed decisions about the deployment and ongoing use of the AI solution. Ensuring the risk register includes all potential risks will highlight the potential gaps in your solution and enable mitigation strategies. Your team can have established roles, responsibilities, and policies for managing AI risks effectively.

    Security

    Each type of AI comes with its own security issues, like adversarial attacks (subtly changing input data to fool the AI), data poisoning (injecting malicious data to corrupt training), and model stealing (recreating a proprietary model). Achieving security in the model performance involves achieving security throughout the solution lifecycle, following a ‘Security by design’ approach to developing your solution. One must secure their data (incoming and outgoing), adopt techniques to protect the AI models, infrastructure, and APIs. While this principle shares its goal with standard security for software solutions, in AI TRiSM this also means making sure your model is not ‘hackable’ in ways specific to AI vulnerabilities.

    How can you adopt AI TRiSM as a company?

    While developers can tackle components individually, companies can also embrace it as a complete framework. The following steps outline how your company can introduce AI TRiSM to your organisation:

    • Adopt AI TRiSM across your entire solution lifecycle (discovery to evaluation), documenting observations and decisions in a final report. A company-wide template standardizes AI TRiSM implementation for all projects.
    • Ensure company-wide awareness of AI TRiSM through clear communication channels. Document model audits and communicate identified risks with proposed mitigation strategies to relevant stakeholders.
    • Provide thorough training and education on AI TRiSM principles and practices across the organization, perhaps through workshops or online modules.
    • Establish partnerships with entities that have a strong focus on AI TRiSM to leverage their expertise and insights.

    Employing TRiSM in AI offers several key benefits, including building trust in AI systems, mitigating potential risks through pre-emptive resolution, ensuring compliance with evolving regulations, and fostering sustainable and transparent AI growth.

    This post has provided an overview of AI TRiSM and its critical role in the responsible development and deployment of AI. In upcoming articles, we will delve deeper into each of the core pillars – Trust, Risk, Security, and Transparency – exploring the specific challenges, techniques, and best practices associated with building trustworthy AI systems.

    Investing in AI TRiSM is an investment in the long-term value and viability of AI. By embedding these principles into our processes, we build a foundation of trust that will be crucial for the continued adoption and positive impact of artificial intelligence!

    Next Steps

    Navigating the complexities of AI TRiSM – ensuring trust, managing risk, and maintaining security – is crucial for successful AI adoption. At Decision Lab, we are committed to developing AI solutions that inherently incorporate these principles from design to deployment.

    Our deep expertise in Explainable AI (xAI) provides the transparency needed for user confidence and regulatory compliance, directly addressing the ‘Trust’ pillar. We specialise in creating effective Human-AI Teaming paradigms, designing systems where human insight complements automated decision-making, ensuring robust operation and essential oversight to mitigate ‘Risk’.

    Furthermore, our development processes are underpinned by strict adherence to rigorous ISO standards, demonstrating our commitment to ‘Security’, reliability, and quality across all our AI solutions. Partner with Decision Lab to build AI systems that are not only high-performing but also fundamentally trustworthy, secure, and aligned with responsible innovation principles.

    To explore how Decision Lab’s AI solutions can benefit your organisation, get in touch. Let’s unlock the full potential of AI.

    The next post in the series Trust in AI Systems: An essential framework for Responsible AI Deployment.

    Author: Anahita Bilimoria, Decision Lab Senior Machine Learning Engineer
    For further updates from Decision Lab, follow us on LinkedIn!