Sector: Critical Infrastructure Security
Origin: Decision Lab’s AI Innovation Lab
Executive Summary
- Delivers 24/7 autonomous situational awareness on standard edge hardware, removing the need for costly server infrastructure refits.
In the high-stakes world of critical infrastructure protection, the gap between a routine patrol and a security breach is often measured in seconds. Yet, traditional surveillance systems are manipulated by intelligent behaviour. Standard algorithms rely on invariant detections (classification of known objects) and often lack the ability to determine action and intent. While these algorithms can effectively detect and classify inanimate objects that might be inherently harmful or broken, they falter when detecting complex actions such as abnormal or harmful patterns of behaviour.
Emerging from our AI Innovation Lab, the Threat-Detection using Autoencoders (T-DA) programme was designed to close this gap. By combining state-of-the-art Computer Vision with novel temporal awareness, we delivered an unsupervised Deep Learning solution capable of learning the ‘pattern of life’ aspects of behaviour. The result is a system that doesn’t just see movement but understands context—differentiating between a scheduled patrol and an unscheduled intrusion without requiring massive, labelled datasets.
The Challenge: The Signal in the Noise
Our client, responsible for the security of high-sensitivity sites, faced a strategic pain point: operational blindness caused by data overload. Their existing surveillance infrastructure relied on simple motion detection and rule-based triggers.
These legacy systems suffered from two critical failures relating to Pattern of Life (PoL):
High False Positive Rate: Innocent environmental changes (e.g., wind-blown debris) or routine scheduled events triggered constant alarms, leading to operator fatigue and desensitisation.
Contextual Blindness: The systems could not distinguish between visually similar but contextually different events. A guard walking a perimeter at 14:00 is routine; a person walking the same path at 03:00 could be a threat. Standard models saw only ‘person walking’.
The client required a solution that could autonomously detect anomalies in real-time, operate on resource-constrained edge devices, and—crucially—learn what normal looks like without needing thousands of manually labelled “threat” examples.
The Solution: Temporal-Aware Deep Learning
Decision Lab deployed a cutting-edge unsupervised anomaly detection pipeline that fundamentally changes how machines perceive security footage.
Traditional systems require training on thousands of examples of threats (which are rare and varied). Instead, we taught the model what normality looks like. By learning the standard pattern of life, the system can autonomously flag any event that deviates—whether it is a known threat type or an entirely new anomaly.
1 . The Core Architecture
To ensure our solution remained modal-agnostic, we experimented with various encoder models, ranging from standard LSTMs to Gaussian Mixture Models. For the Proof of Concept (PoC), we implemented a Vision Transformer (ViT)-based Variational Autoencoder (VAE).
- The Encoder (ViT): Unlike standard CNNs that look at localised pixels, the ViT uses self-attention mechanisms to capture global contextual information from video frames.
- The Decoder: This component attempts to reconstruct the video frames from the encoder’s summary.
- The Trigger: If the model cannot accurately reconstruct a scene (resulting in a high reconstruction error), it indicates the event is not in its learned database of normal behaviours, instantly triggering an anomaly alert.
2. Innovation: Temporal Integration
Standard computer vision models are time-blind; they see a person walking but do not know if it is 14:00 (routine) or 03:00 (suspicious). To solve this, we engineered a novel Cyclic Time Encoding mechanism.
- Cyclic Encoding: We encoded timestamps using sine and cosine functions. This captures the periodic nature of time (24-hour cycles) more effectively than linear numbers.
- Contextual Conditioning: This time vector modulates the model’s latent space. effectively teaching the AI that Activity A is normal at Time X, but anomalous at Time Y.
This approach yielded two critical capabilities:
- Temporal Anchoring: We successfully introduced a temporal factor—contextual metadata that anchors the model in time, rather than relying solely on visual pixel data.
- Scalable Context: While this PoC used timestamps, the architecture can ingest any form of metadata. The model can be conditioned on geographical data (weather, pressure), or solution-specific constraints (security clearance levels, job titles), making T-DA highly adaptable across defence, supply chain, and rail verticals.
3. Edge Deployment (SWaP Optimised)
Meeting the strict requirements for defence operations, we optimised the model using FP16 (16-bit floating point) precision.
- Size Reduction: This compressed the model size by 50%.
- Performance: The system runs efficiently on resource-constrained edge devices (e.g., drones, remote sentries) without sacrificing detection accuracy.
- Security: Data is processed locally, reducing bandwidth requirements and closing the attack surface associated with cloud transmission.
Results & Impact
The T-DA project successfully demonstrated that autonomous systems can reduce the cognitive load on human operators while enhancing threat detection.
- High Precision: The ViT-based VAE achieved an ROC AUC of 0.855 on general visual anomaly detection, validating the unsupervised approach.
- Operational Efficiency: The move to FP16 precision resulted in a 50% reduction in model size and significant runtime memory savings, enabling deployment on standard edge hardware rather than expensive server racks.
Reduced Fatigue: By automating the detection of contextually specific anomalies, the system significantly reduced the time security personnel spent reviewing false alarms, allowing them to focus on genuine threats.
FOCUS: AI TRiSM (Trust, Risk, and Security Management)
As part of Decision Lab’s commitment to Responsible AI (read our full series here), the T-DA project was developed in strict alignment with the AI TRiSM framework. In high-stakes defence environments, an AI model must be as trustworthy as the officers using it.
1. Trust: Explainability beyond the Black Box
A security operator cannot act on an alert they don’t understand. We moved beyond simple “anomaly scores” by integrating Explainable AI (XAI) techniques.
- Heatmaps: The system provides real-time reconstruction error heatmaps, visually highlighting exactly where in the frame the anomaly is occurring (e.g., highlighting a specific backpack or unauthorised vehicle).
- Contextual Logic: We explored the integration of LLMs to generate natural language explanations, translating complex vector data into clear summaries: ‘Unusual activity detected: Person running at 02:45 AM (high deviation from routine).’
2. Risk: Proactive Reliability
Unsupervised models can drift over time if the environment changes. We mitigated this risk through:
- Synthetic Anomaly Injection: To rigorously test the system before deployment, we developed a methodology to inject synthetic temporal anomalies into the data, ensuring the model could catch threats that hadn’t happened yet.
- Bias Audits: We conducted formal audits of the training data to ensure the normal baseline didn’t inherently bias the model against specific demographics or harmless behaviours.
3. Security: ModelOps & Data Integrity
Security is paramount not just in the physical site, but in the digital pipeline.
- Data Protection: We implemented encrypted channels for all video streams and strict access controls for training data.
- ModelOps: A robust lifecycle management framework was established, including version control for model weights and automated drift detection to trigger retraining. This ensures the model adapts to new patterns of life securely and transparently.
Learn more in our AI TRiSM blog series.
Conclusion
The T-DA project illustrates the power of the Decision Lab Innovation Lab to translate theoretical AI advances into robust, deployable security solutions. By treating time as a critical feature of reality, we moved surveillance from reactive monitoring to proactive threat detection.
Project T-DA Key Facts:
- Developer: Decision Lab
- Primary Tech: Vision Transformer (ViT) & Variational Autoencoder (VAE)
- Innovation: Cyclic Time Encoding (Temporal Awareness)
- Use Case: Unsupervised anomaly detection for critical infrastructure.
- Performance: 0.855 ROC AUC with 50% model compression via FP16.
Would you like to explore how Decision Lab can streamline your operations? Contact our Innovation Team today.