ADR 004 — LINSTOR + DRBD for Distributed Storage

Synced read-only from /home/synnet/mirrors/synvirt-product/docs/decisions/004-linstor-drbd-for-distributed-storage.md. Edit at the source, not here.

ADR 004 — LINSTOR + DRBD for Distributed Storage

Status: Accepted (integration in Hito 3; enabled in Twin mode) Date: 2026-04-18 Deciders: SYNNET Engineering

Context

SynVirt’s Twin and Cluster modes require replicated block storage so that:

Live migration is possible: a VM’s disk must be accessible from both the source and destination node simultaneously (or the migration must be storage-live, meaning the storage follows the VM).
HA failover works automatically: if a node fails, another node can immediately boot that node’s VMs because the storage is already replicated and up-to-date.
No shared SAN is required: SMBs and MSPs in Mexico and LATAM rarely have a dedicated SAN. The solution must work with standard servers and direct-attached disks connected by a 10 GbE storage link.

LINSTOR + DRBD integration itself lands with module-storage in Hito 3 (June 2026). The replication is wired up when Twin mode ships in October 2026. Hito 0.3 through Hito 2 use local storage only (ZFS localdata without DRBD replication).

Decision

Use DRBD (Distributed Replicated Block Device) for synchronous block replication between nodes, orchestrated by LINSTOR (LinStor Resource Orchestrator). VM volumes are stored as ZFS zvols (see ADR 003) that DRBD replicates over a dedicated storage network link.

Rationale

DRBD is battle-tested for synchronous replication: DRBD has been in production Linux deployments since 2001 and is the replication layer underlying LINBIT’s commercial products. Synchronous replication (Protocol C) guarantees that a write is not acknowledged to the guest until it has been committed on both nodes, eliminating data loss on failover.
LINSTOR provides orchestration without SAN complexity: LINSTOR manages DRBD resources as a pool, handling volume creation, deletion, and replication topology. This gives us the operational model of a SAN (centrally managed storage volumes) without the hardware cost or vendor lock-in of a physical SAN.
No shared storage requirement: The DRBD+LINSTOR model requires only two servers connected by a network link. There is no need for a shared disk array, Fibre Channel, or iSCSI SAN. This is critical for our SMB target market where rack space and budget are constrained.
ZFS zvol integration: ZFS zvols expose a raw block device. DRBD replicates at the block level, so the ZFS checksum layer on each node independently verifies data integrity. This provides end-to-end integrity: even if a bit flips during network transmission, DRBD’s checksum mechanism or ZFS on the receiving side will detect it.
Operator-familiar technology: DRBD is the replication layer underneath several open-source HA storage stacks our target operators may already have encountered. Reusing a well-known building block keeps the conceptual gap small for sysadmins coming from other small-cluster virtualization platforms.
LINSTOR CSI integration path: LINSTOR has a CSI driver for Kubernetes. While out of scope, this provides a future integration path if customers want to run Kubernetes inside SynVirt and expose replicated storage to container workloads.

Consequences

Complex setup: Configuring DRBD resources, LINSTOR satellites, and the storage network requires careful network planning. The Twin mode setup wizard must abstract this complexity from operators.
DRBD kernel module dependency: DRBD is a kernel module installed via DKMS (similar to ZFS, ADR 003). Kernel upgrades require DKMS rebuild verification for both ZFS and DRBD.
Dedicated storage network required: Synchronous DRBD replication adds write latency proportional to the round-trip time between nodes. A dedicated 10 GbE link between nodes is required for acceptable VM disk performance. Mixed storage+management networks are not supported in Twin mode.
Minimum hardware requirements increase: Twin mode requires two servers each with at minimum 10 GbE NICs and sufficient RAM for ZFS ARC plus DRBD metadata. Twin mode hardware requirements will specify this.
LINSTOR controller placement: In Twin mode, LINSTOR needs a controller node. We will embed it in synvirt-daemon to avoid a separate process. In a two-node setup, the witness node can host the LINSTOR controller as a tiebreaker.

Alternatives Considered

Ceph

Pros: Battle-tested distributed storage, wide community, used by OpenStack and many large deployments.
Cons: Ceph is designed for clusters of 3+ nodes minimum (a 2-node Ceph cluster without a monitor quorum is not HA). The operational complexity and resource overhead (Ceph recommends 1 GB RAM per OSD daemon) is disproportionate for a 2-node SMB deployment. Ceph’s minimum viable configuration is too heavy for the Twin mode use case.
Decision: Rejected for Twin mode. May be revisited as an optional backend for large Cluster deployments post-GA.

NFS (shared via a NAS appliance)

Pros: Simple, widely supported, no kernel module.
Cons: NFS provides no HA by itself. If the NFS server fails, all VMs on all nodes fail. NFS over a LAN introduces latency that makes synchronous writes impractical. This approach recreates the SAN problem (a single storage appliance is a single point of failure) rather than solving it.
Decision: Rejected. Fundamentally incompatible with the HA requirement.

Proprietary 2-node HA storage appliances

Pros: Purpose-built for small HA deployments, simple setup, turnkey.
Cons: Closed-source with per-host licensing, must be embedded under a commercial agreement, and introduces a redistribution dependency on a third party. Incompatible with the product’s goals of self-contained deployment and transparent open-source foundations.
Decision: Rejected. Licensing and redistribution constraints are disqualifying; an open-source stack we can fully own is required.