The event bus

VECTOR archive › Architecture › The event bus

The governance stack is described as a chain, and a chain implies each stage calling the next. It does not. The protocols never call each other; they publish and subscribe through a shared asynchronous bus, and that indirection is load-bearing.

Problem

A chain of direct calls couples every stage to the stage after it. Detection has to know that classification exists, classification has to know about containment, and any change to the ordering is a change to every component in it.

Worse for this system: a direct call inherits the callee's latency. If containment is slow, detection is slow, because detection is blocked inside it — and detection is the stage whose timeliness everything else depends on.

Context

The stack has nine protocols with a defined execution order, a hard per-cycle time budget, and a requirement that "something is wrong" reaches its consumers faster than ordinary telemetry does. Several components consume the same events — Sentinel's anomaly stream is read by classification, by the intelligence layer's trigger conditions, and by the audit record.

Constraints

Delivery must be bounded: an unbounded queue under sustained overload converts a latency problem into a memory problem, which fails later and worse. Ordering has to be preserved where it is meaningful. And priority must be expressible, because the whole point is that alarms are not ordinary traffic.

Alternatives considered

Direct calls. Simplest, and rejected for the coupling and latency inheritance above.

A plain FIFO queue. Decouples the stages and cannot express urgency. An anomaly published behind a backlog of routine metrics is delivered after that backlog — and the backlog is longest exactly when the anomaly matters most. This is the failure mode the design specifically rejects.

Chosen architecture

AsyncEventBus in core/event_bus.py: an asyncio.PriorityQueue with a bounded maximum size, topic-based subscribe, and events carried as a typed Event wrapped in an internal _PrioritizedEvent for ordering.

Two properties follow. Priority is a first-class attribute of publication rather than a convention, so a high-priority anomaly overtakes queued telemetry by construction. And the queue is bounded, so overload manifests as backpressure at a known limit rather than as unbounded growth.

Topic-based subscription is what makes one anomaly stream serve several consumers without any of them knowing the others exist.

Subsystem relationships

Every protocol in the governance stack is a publisher, a subscriber, or both. Sentinel is the primary publisher and subscribes to nothing, which is why it keeps working when everything above it has failed. The intelligence layer's triggers are evaluated against bus events rather than against protocol internals.

Data flow

Sentinel ──publish(metrics, medium)──┐
         └─publish(anomaly, high)────┤
                                     ▼
                              AsyncEventBus
                                     │  (priority ordered, bounded)
                 ┌───────────────────┼───────────────────┐
                 ▼                   ▼                   ▼
            Serpentine        intelligence triggers   audit record

Tradeoffs

Indirection costs traceability. A direct call stack shows you what invoked what. A bus does not, and reconstructing a causal chain afterwards requires the events to carry enough context to do it. This is part of why decisions carry their own provenance — see explainability before automation.

Priority can starve. A sustained flood of high-priority events delays medium-priority ones indefinitely. Acceptable here because a sustained anomaly flood is itself a condition the stack is designed to escalate on, but it is a real property rather than an oversight.

Bounded queues drop. At the limit, something is refused. Refusal at a known boundary is preferable to unbounded growth, and it is still a loss.

Failure modes

Queue saturation. Backpressure at max_queue_size. Bounded and observable.

Subscriber slower than publisher. The queue grows toward its bound; the slow consumer is the fault, and the bus makes it visible rather than absorbing it silently.

A handler raising. Whether one failing subscriber affects delivery to others is not evident from the structure alone, and this document does not assert an answer.

Future evolution

The natural direction is per-topic backpressure policy — allowing telemetry to be shed under load while alarms are never dropped. That is inference about a sensible next step, not a documented plan.