Daemon split — Updater & Migration into standalone daemons

Synced read-only from /home/synnet/mirrors/synvirt-product/docs/daemon-split/DESIGN.md. Edit at the source, not here.

Daemon split — Updater & Migration into standalone daemons

Status: design (feature/daemon-split). Author: autonomous engineering run, 2026-06-09.

This document is the design contract for extracting two subsystems out of synvirt-daemon into standalone daemons:

• synvirt-updaterd — owns the ISO-based transactional update flow (preflight, boot-environment checkpoint, apply transaction, reboot, rollback, post-verify). Rationale: a transactional update must be able to stop synvirt-daemon, apply a signed update, restart it, and roll back, so the orchestrator cannot share the daemon’s process or lifecycle.
• synvirt-migrationd — owns VM migration jobs (live/offline between SynVirt hosts). Rationale: a long-running migration must survive a synvirt-daemon restart. Same philosophy as Fix70 (a control-plane restart never harms guests) — extended so a control-plane restart never kills an in-flight migration either.

synvirt-daemon remains the single public API surface on TLS :443 and proxies updater/migration requests to the sub-daemons over Unix domain sockets. If a sub-daemon socket is unavailable, the daemon returns the standard error envelope (503 Service Unavailable) — it never panics.

---

0. Scope clarification — migration vs. the migrator crate

The existing crates/migrator (synvirt-migrator, mounted at /api/v1/migrator) is the VM import engine: it pulls a VM from an external hypervisor control plane (vim/wmi/pve/prism/xen/libvirt-remote adapters) or imports a portable archive/disk image, and lands it on a local DestinationSink. That is conversion/ingest — a different concern from host-to-host migration.

A repo-wide search for host-to-host live migration primitives (migrateToURI, virsh migrate, virDomainMigrate) returns only comments — there is no existing host-to-host migration code, no job tracking for it, nothing to move. synvirt-migrationd is therefore greenfield: built per the spec (SQLite-persisted job model + executor trait + mock), with the real libvirt executor stubbed and documented as the next step (no real migration is executed by this work — execution is behind a trait, tests use a mock).

The migrator import engine is out of scope and is left untouched.

---

1. Target crate layout

Three new workspace members:

crates/synvirt-ipc/          # lib   — generic HTTP-over-UDS transport
crates/synvirt-updaterd/     # lib+bin — updater orchestrator + UDS API
crates/synvirt-migrationd/   # lib+bin — migration jobs + UDS API

Each daemon crate is lib + bin: the binary is the daemon; the library exposes the wire DTO types and a typed client so synvirt-daemon can build its OpenAPI surface and proxy without duplicating type definitions. This honours the domain-ownership invariant (the type lives next to its owner) while keeping the daemon’s generated OpenAPI byte-stable.

1.1 synvirt-ipc (generic transport)

No domain knowledge. Pure plumbing reused by both sub-daemons and the daemon proxy:

• server::bind_uds(path, mode) -> UnixListener — unlinks any stale socket, creates the parent dir with 0700, binds, and chmods the socket to a restrictive mode (default 0600). Loopback-only by construction (a UDS has no network surface); the filesystem permission is the access control.
• server::serve(listener, router, shutdown) — serves an axum::Router over the UnixListener with graceful shutdown on a future.
• client::UdsClient { socket_path } — sends one HTTP/1.1 request over a fresh UnixStream (hyper 1.x client conn + hyper-util TokioIo). Two entry points: request_buffered (await full body) and request_streaming (return a streaming body, used for SSE proxying).
• Errors: IpcError (connect/io/protocol). The daemon maps a connect failure to its own 503 envelope; synvirt-ipc has no opinion on error-body shape (keeps it daemon-agnostic).

Transport choice rationale: the repo has no existing daemon↔daemon local-IPC pattern (the cluster mesh is tonic-over-TCP+rustls; the guest agent is a custom length-prefixed framing over UDS). Per the spec we pick the boring option — HTTP/1.1 over UDS with axum on the server side and hyper on the client side. axum, hyper, hyper-util, and tower are already in the workspace lock, so no new heavy dependency.

1.2 synvirt-updaterd

Library modules (moved behavior-preserving from crates/daemon/src/update_v2/):

Module	Origin	Notes
state	update_v2/state.rs	UpdateState machine — verbatim
progress	update_v2/progress.rs	ProgressEvent (SSE) — verbatim
error	update_v2/error.rs	OrchestratorError — verbatim
host_info	update_v2/host_info.rs	live host probe — verbatim
orchestrator	update_v2/orchestrator.rs	actor — verbatim
http	crate/api/update_v2.rs	axum router + OrchestratorError→envelope map
envelope	new	minimal error envelope serializing identically to the daemon’s ErrorEnvelope
config	new	UpdaterdConfig (socket path, iso library dir, BE pool, audit path)

The orchestrator is already self-contained: it depends only on synvirt-iso and synvirt-update plus its own sibling modules. It does not touch AppState, the daemon’s activity log, or the daemon’s SSE event bus (it owns its own tokio::sync::broadcast channel and its own synvirt_update::AuditLog). This is what makes the extraction a move, not a redesign.

Binary (main.rs): load config → construct Orchestrator (Library + BootEnvManager + AuditLog + running release) → run the post-verify resume task (moved verbatim from daemon/src/main.rs:1022) → bind the http router to the UDS → wait for SIGTERM → graceful shutdown.

1.3 synvirt-migrationd

Greenfield library modules:

Module	Responsibility
model	Job, JobState (queued→preparing→running→completed|failed|cancelled|interrupted), JobKind (live|offline), request/response DTOs
store	SQLite (rusqlite) persistence at /var/lib/synvirt/migrationd.db; schema migration on open; CRUD + state transitions
executor	MigrationExecutor trait; MockExecutor (tests); LibvirtExecutor stub (documented next step — no real migration)
manager	JobManager: create/list/get/cancel + state machine + startup reconcile
error	MigrationError with E_* discriminants
http	axum router (create/list/get/cancel) + error→envelope map
envelope	same minimal envelope type as updaterd
config	MigrationdConfig (socket path, db path)

Binary (main.rs): load config → open store → reconcile (mark any job left in a non-terminal state — queued/preparing/running — as interrupted; never act on a guest at startup) → bind UDS → SIGTERM.

---

2. UDS IPC contract

Both sub-daemons speak HTTP/1.1 over their UDS. The daemon proxies the public routes 1:1 onto the sub-daemon socket, preserving method, the path suffix, query string, request body, and response status/body. SSE bodies are streamed, not buffered.

2.1 Updater (unchanged public surface, now proxied)

Public routes (preserved verbatim — same paths, operation_ids, bodies):

POST /api/v1/system/update/select-iso      -> updaterd POST /select-iso
POST /api/v1/system/update/scan            -> updaterd POST /scan
POST /api/v1/system/update/preflight       -> updaterd POST /preflight
GET  /api/v1/system/update/plan            -> updaterd GET  /plan
POST /api/v1/system/update/checkpoint      -> updaterd POST /checkpoint
POST /api/v1/system/update/apply           -> updaterd POST /apply
POST /api/v1/system/update/reboot          -> updaterd POST /reboot
POST /api/v1/system/update/rollback        -> updaterd POST /rollback
GET  /api/v1/system/update/status          -> updaterd GET  /status
GET  /api/v1/system/update/history         -> updaterd GET  /history
GET  /api/v1/system/update/logs    (SSE)   -> updaterd GET  /logs   (streamed)

The update-ISO library CRUD (/api/v1/system/update/isos*, update_isos.rs) stays in the daemon: it is plain filesystem management with no daemon-lifecycle dependency, and synvirt_iso::Library holds no exclusive lock (no sled/sqlite), so the daemon and updaterd can both read the same library directory safely. Keeping it daemon-side minimizes moved surface and risk.

host_info is gathered inside updaterd (it probes the live host), so the daemon no longer needs gather_host_info.

2.2 Migration (additive — new routes, OpenAPI-native)

New public routes (snapshot-panel pattern, additive to OpenAPI):

POST   /api/v1/system/migration/jobs        -> migrationd POST /jobs      (create)
GET    /api/v1/system/migration/jobs        -> migrationd GET  /jobs      (list)
GET    /api/v1/system/migration/jobs/{id}   -> migrationd GET  /jobs/{id} (get)
POST   /api/v1/system/migration/jobs/{id}/cancel -> migrationd POST /jobs/{id}/cancel

Create body: { vm, source_host, target_host, kind: "live"|"offline" }. Job response: { id, vm, source_host, target_host, kind, state, created_at, updated_at, detail }.

2.3 Health

Each sub-daemon serves GET /healthz → 200 {"status":"ok", "service":"...","version":"..."} for the integration harness and future readiness probes. Not exposed publicly.

---

3. Failure modes

• Sub-daemon socket missing / refused → daemon proxy returns 503 with ErrorEnvelope { code: "E_UPDATER_UNAVAILABLE" / "E_MIGRATION_UNAVAILABLE", message: "..." }. Never a panic, never a 5xx hang.
• Sub-daemon crash mid-request → hyper read error → same 503 envelope.
• Updater orchestrator busy → 409 E_UPDATE_BUSY (produced by updaterd, forwarded verbatim).
• Migration job not found → 404 E_MIGRATION_JOB_NOT_FOUND.
• SQLite open failure (migrationd) → daemon start fails fast with a clear journald error (the DB is the source of truth; degrading silently would lose jobs).
• Stale socket file after an unclean shutdown → bind_uds unlinks it before binding.

---

4. migrationd restart-safety

The job state is persisted in SQLite at /var/lib/synvirt/migrationd.db. On start, reconcile():

1. Selects every job whose state is non-terminal (queued/preparing/running).
2. Transitions each to interrupted with a detail of “interrupted by migrationd restart” and bumps updated_at.
3. Never touches a guest, source, or target at startup — reconcile is bookkeeping only. Operators (or a future supervisor) decide whether to re-create an interrupted job.

This mirrors the Fix70 guarantee: lifecycle events of the control plane never mutate the data plane implicitly.

---

5. Config keys + defaults

Product-convention paths (/etc/synvirt, /var/lib/synvirt, /run/synvirt) are exempt from the “no hardcoded operator-changeable value” rule, but every one below is still overridable.

Daemon (DaemonConfig, client side)

Key	Default	Meaning
updater.enabled	true	mount + proxy the update routes
updater.socket_path	/run/synvirt/updaterd.sock	updaterd UDS
migration.enabled	true	mount + proxy the migration routes
migration.socket_path	/run/synvirt/migrationd.sock	migrationd UDS

synvirt-updaterd (UpdaterdConfig)

Loaded from /etc/synvirt/updaterd.toml (override: SYNVIRT_UPDATERD_CONFIG); each field also has an env override.

Key	Default	Meaning
socket_path	/run/synvirt/updaterd.sock	bind path
socket_mode	0o600	socket permission
iso_library_dir	synvirt_iso default	update-ISO library
boot_env_pool	synvirt-root	ZFS BE pool name
audit_path	synvirt_update default	audit log path
release	env!("SYNVIRT_RELEASE")	running version string

synvirt-migrationd (MigrationdConfig)

Loaded from /etc/synvirt/migrationd.toml (override: SYNVIRT_MIGRATIOND_CONFIG).

Key	Default	Meaning
socket_path	/run/synvirt/migrationd.sock	bind path
socket_mode	0o600	socket permission
db_path	/var/lib/synvirt/migrationd.db	SQLite job store

---

6. systemd unit design (files in repo only — NOT installed)

Both units are written to the repo packaging location iso-builder/live-build/config/includes.chroot/etc/systemd/system/. They are not installed, enabled, or started by this work.

synvirt-updaterd.service

• Type=notify-free simple long-running socket server (Type=simple).
• Independent of synvirt-daemon’s lifecycle: no After/Before binding to synvirt-daemon.service, no PartOf/BindsTo. It only needs /run/synvirt to exist. This is the structural reason the updater can stop/restart the daemon without taking itself down.
• After=local-fs.target; Wants=/Requires= nothing on the daemon.
• Hardening mirrors synvirt-network.service: ProtectSystem=strict, ReadWritePaths=/etc/synvirt /var/lib/synvirt /run/synvirt, NoNewPrivileges, restrictive CapabilityBoundingSet. The apply phase needs mount/umount/systemctl reboot, so the production unit keeps CAP_SYS_ADMIN and runs as root (documented; tightening is follow-up).

synvirt-migrationd.service

• Type=simple long-running socket server.
• After=libvirtd.service (per spec — the real executor will drive libvirt). Ordering-only: no PartOf/BindsTo/Requires on libvirtd or the daemon, so it never gets dragged down by a daemon restart.
• Must not conflict with synvirt-guest-drain.service: that unit is an ExecStop-only host-shutdown drain ordered After=libvirtd.service synvirt-daemon.service. migrationd adds no Conflicts= and shares no PartOf relationship with it, so the two are independent.
• Hardening mirrors the other planes; ReadWritePaths includes /var/lib/synvirt (the SQLite DB) and /run/synvirt (the socket).

Boot ordering summary:

openvswitch-switch → synvirt-network → libvirtd → synvirt-daemon
                                          └→ synvirt-migrationd (After=libvirtd, ordering-only)
synvirt-updaterd: independent (After=local-fs.target only)
synvirt-guest-drain: ExecStop-only, After=libvirtd synvirt-daemon (unchanged)

---

7. OpenAPI stability

• Updater routes: the daemon keeps the exact #[utoipa::path] handler functions (same operation_ids, same response body = types) — only the handler bodies change from in-process calls to UDS proxy calls. The DTO types move to synvirt-updaterd but keep identical type names + fields + serde/ToSchema derives, so the generated component schemas are byte-identical. api/openapi.rs swaps its imports from crate::update_v2::… to synvirt_updaterd::…. The committed crates/web-ux-v2/openapi.snapshot.json is re-dumped and diffed; the update portion must be unchanged.
• Migration routes: additive. New #[utoipa::path] handlers + new ToSchema types registered in api/openapi.rs. The snapshot grows by exactly the migration surface.

---

8. Build sequence (phases)

1. Scaffold the three crates (build green, smoke tests).
2. Extract updater → synvirt-updaterd; daemon proxies; remove dead code; verify OpenAPI update portion unchanged.
3. Implement synvirt-migrationd (TDD: store, manager, reconcile, executor mock); daemon proxies additive routes.
4. Integration harness: spawn both sub-daemons as plain processes and round-trip through the daemon proxy.
5. cargo fmt + clippy across new/moved code; full suite green.

Each phase is an atomic commit with explicit file paths, committed only when the full test suite is green.