The simulation layer

VECTOR archive › Architecture › Simulation

A control policy for traffic infrastructure cannot be learned on traffic infrastructure. The exploration required to learn anything is precisely what must never happen on a live road, which makes simulation a requirement rather than a convenience.

Problem

Learned components need many more interactions than reality can safely provide, and the interactions they need most are the unusual ones. A system that only ever observes normal operation has no basis for behaving well outside it.

Context

SUMO — Simulation of Urban Mobility — is an established open-source traffic simulator. Using it rather than building a simulator means inheriting a model of traffic that was not written to make this system look good, which is a real epistemic advantage.

Constraints

The interface the simulator presents has to be the same shape the system sees in production, or the policy learns against an environment it will never meet. Rollouts must be recordable for training. And the dependency has to be replaceable — binding the system to one simulator is the coupling problem again, one layer out.

Alternatives considered

A purpose-built simulator. Full control, and it would encode the same assumptions as the system being tested. A simulator written by the author of the controller tends to reward the controller.

Recorded traffic data replayed. Real, and non-interactive: it cannot answer what would have happened had the system acted differently, which is the entire question.

Chosen architecture

BaseSimulationAdapter in simulation/sumo_adapter.py defines a familiar reinforcement learning environment contract — reset, get_state, get_valid_actions, step — with SumoAdapterConfig carrying configuration and the SUMO binding behind it.

get_valid_actions is worth noting. The environment states which actions are permissible rather than leaving the agent to propose something invalid and be corrected, which is the same relationship the policy layer has to the decision path in the live system: constraints available during the decision rather than as a veto afterwards. See governance before intelligence.

Rollouts are captured as TransitionRecord and exposed as a RolloutDataset, so simulated experience enters training through a typed boundary rather than an ad-hoc one.

Subsystem relationships

Produces training data for the learned components in learning/, entering training through the same typed boundary that historical data does — see the data pipeline. Mirrors the contract of the hardware abstraction layer, which is what allows the same decision path to run against a simulator or a controller.

Data flow

reset → get_state → get_valid_actions → (policy) → step → TransitionRecord
   ↑                                                            │
   └──────────────────── episode ends ──────────── RolloutDataset → training

Tradeoffs

A simulator is a model, and every model is wrong somewhere. Policies learned in simulation encode its inaccuracies. This is the central limitation and no amount of simulation fidelity removes it.

Simulated experience is cheap, which makes it tempting to trust. Large volumes of simulated data produce confident estimates about a world that is not the one the system will operate in.

Failure modes

Reality gap. Behaviour that is correct in simulation and wrong in deployment. The runtime defence is not better simulation — it is that VIHAAN narrows autonomy when conditions stop matching expectations.

Overfitting to the scenario set. A policy excellent on the scenarios it was trained against and undistinguished elsewhere.

Future evolution

A pilot against real infrastructure is the outstanding item, and it is the only thing that can establish what simulation cannot — noted as unfinished in the v1 changelog. A digital twin, running alongside the live system rather than ahead of it, is the longer-term direction.