Adversarial robustness

VECTOR archive › Architecture › Adversarial robustness

Sentinel detects values that are statistically unusual. That is the correct defence against a sensor that is broken. It is no defence at all against a sensor that is being manipulated, because manipulated inputs are constructed to be unremarkable.

Problem

Infrastructure that responds to observation can be influenced by anyone who can influence the observation. An adversary does not need access to the decision system if they can feed it plausible, false readings — and plausibility is exactly what defeats threshold and z-score detection.

Context

This was identified as a gap before it was addressed, and named as future work in the published account of v1. It now exists as a subsystem, which is worth stating plainly: the roadmap item has been built.

Constraints

It has to operate on the same inputs the live system consumes, cheaply enough to sit in a real-time path, and without a labelled corpus of attacks — because the attacks that matter are the ones nobody has catalogued.

Alternatives considered

Wider statistical thresholds. Catches cruder manipulation and, by construction, misses anything shaped to look normal. It also raises false positives, which erodes confidence in the whole chain.

Supervised attack classification. Effective against known attack classes, and it requires examples of attacks. A defence that only recognises what it has been shown is not a defence against a deliberate adversary.

Chosen architecture

security/adversarial_robustness.py, built around a RobustnessAutoencoder with a RobustnessTrainer, RobustnessConfig and RobustnessTrainingConfig, exposed at runtime as an AdversarialRobustnessGuard.

The autoencoder approach answers the labelled-data constraint. An autoencoder is trained to reconstruct normal inputs and is scored by how badly it fails: an input drawn from the distribution it learned reconstructs well, and an input constructed adversarially reconstructs poorly, because it is not from that distribution however plausible its individual values look.

This detects unfamiliarity rather than abnormality, which is a genuinely different question from the one Sentinel asks — and the reason both exist.

SecurityDataset and collate_security_samples provide the training path; the Guard is the runtime boundary.

Subsystem relationships

Sits alongside detection rather than inside it. Where Sentinel asks "is this value unusual for this system", the guard asks "does this input resemble anything I was trained on". Its output belongs upstream of the confidence that observation before action requires every observation to carry.

Data flow

input → RobustnessAutoencoder → reconstruction error → AdversarialRobustnessGuard
                                                     → trust signal on the observation

Tradeoffs

Legitimate novelty is indistinguishable from attack. A genuinely unprecedented but real condition reconstructs poorly, for the same reason a crafted input does. The guard cannot tell them apart, and both narrow autonomy — the safe direction, and a real cost during unusual legitimate events.

The defence inherits its training distribution. An autoencoder trained on a period of normality treats drift away from that period as suspicious.

A learned defence has its own adversarial surface. An adversary who knows the mechanism can, in principle, construct inputs that reconstruct well and are still false. This is an honest limitation of the approach, not a defect of this implementation.

Failure modes

False positives during real anomalies — a legitimate emergency is unusual, and unusual is what the guard reports on.

Silent staleness. A guard whose training distribution no longer describes the system degrades gradually and without announcing it.

Future evolution

Periodic retraining against recent normal operation is the obvious requirement, and it trades directly against the guard being able to notice slow drift — the same tension Sentinel has with its rolling window. Whether the current implementation retrains is not established by structure alone.