Skip to content
SYNNET DevHub

Daemon split — final report

Synced read-only from /home/synnet/mirrors/synvirt-product/docs/daemon-split/REPORT.md. Edit at the source, not here.

Daemon split — final report

Section titled “Daemon split — final report”

Branch: feature/daemon-split (worktree at /home/synvirt/worktrees/daemon-split, branched from 0.15.0 @ cfd9061). Overnight unattended run, 2026-06-09.

Outcome: all six phases completed and green. The updater and migration subsystems are now standalone daemons (synvirt-updaterd, synvirt-migrationd) on private Unix domain sockets, with synvirt-daemon proxying their public routes on TLS :443. The main checkout (/home/synvirt/dev) and all live hosts were untouched.

Per-phase summary

Section titled “Per-phase summary”
Phase Result
0 — Preflight & isolation ✅ 30 GB free; worktree created from cfd9061; baseline green
1 — Investigation + design ✅ DESIGN.md committed
2 — Scaffold ✅ 3 new crates build + smoke tests; units in repo
3 — Extract updater → updaterd ✅ behavior-preserving move + transparent proxy; OpenAPI preserved
4 — synvirt-migrationd ✅ greenfield SQLite job engine + executor trait/mock + additive proxy
5 — Integration polish ✅ real-process harness; fmt + clippy on new/moved code
6 — Report ✅ this file

Mission decision: migration ≠ the migrator crate

Section titled “Mission decision: migration ≠ the migrator crate”

A repo-wide search for host-to-host live-migration primitives (migrateToURI, virsh migrate, virDomainMigrate) found only comments. The existing crates/migrator is the foreign VM import engine (vim/wmi/pve/prism/xen adapters, mounted at /api/v1/migrator) — a different concern. synvirt-migrationd (host-to-host migration, vMotion- style) was therefore built greenfield and the import engine left untouched.

Commits (newest first)

Section titled “Commits (newest first)”
d8fbe23 feat(daemon-split): implement synvirt-migrationd + daemon migration proxy
ab6fb6d refactor(daemon-split): extract update orchestrator into synvirt-updaterd
7307130 feat(daemon-split): scaffold synvirt-ipc, synvirt-updaterd, synvirt-migrationd
25c24cb docs(daemon-split): design + progress for updater/migration split

(A Phase-5/6 docs commit adds the harness + this report.)

Base: cfd9061 (“docs: cluster-phase2 consolidated into 0.15.0 mainline”). Net change: 45 files, +3575 / −368.

Baseline vs final test counts

Section titled “Baseline vs final test counts”
  • Baseline (Phase 0), cargo test --workspace --exclude synvirt-guest-installer --exclude synvirt-guest-installer-rs: 1998 passed / 0 failed / 4 ignored (136 binaries).
  • Final (Phase 5), same command: 2028 passed / 0 failed / 4 ignored — fully clean, including the real-process integration harness (sub_daemons_round_trip_through_the_proxy ... ok).

New tests added by this work: synvirt-ipc 5, synvirt-updaterd 12 (state-machine + config + host_info, moved + kept), synvirt-migrationd 18 (model + store + manager), daemon proxy 3 (fake-UDS), daemon integration harness 1 (real sub-daemon processes).

The prompt’s “~388–396 passing” estimate was a narrower subset; the full workspace suite is far larger and unambiguously green.

Environment caveats (not code issues)

Section titled “Environment caveats (not code issues)”
  1. synvirt-guest-installer / -rs excluded — their build.rs requires 131 external SHA-256-pinned guest payloads (virtio-win drivers, qemu-ga.msi) that are never committed (CLAUDE.md §5.8) and are absent on this build box. Unrelated to the daemon split.
  2. crates/live fonts — build.rs needs gitignored JetBrains Mono TTFs; copied from the main checkout into the worktree (read-only from dev, still gitignored, never committed) so the workspace builds.
  3. synvirt-webrtc SVT-AV1 FFI — one lib test intermittently SIGSEGVs inside the SVT-AV1 encoder under load; it passes on isolated re-run (cargo test -p synvirt-webrtc --lib → 104/0). Pre-existing, environment-related, unrelated to this work.

Files created / moved / deleted

Section titled “Files created / moved / deleted”

New crates

  • crates/synvirt-ipc/ — generic HTTP-over-UDS transport (lib).
  • crates/synvirt-updaterd/ — update daemon (lib + bin).
  • crates/synvirt-migrationd/ — migration daemon (lib + bin).

Moved (git renames, history preserved) — daemon → updaterd: update_v2/{state,progress,error,host_info,orchestrator}.rs.

Deleted: crates/daemon/src/update_v2/mod.rs.

Daemon edits: api/update_v2.rs (now a transparent proxy), api/migration.rs (new, proxy), api/uds_proxy.rs (new, shared proxy + harness), api/mod.rs, api/health.rs (AppState), api/openapi.rs, config/mod.rs, main.rs, web/routes.rs, Cargo.toml.

systemd units (in repo only, NOT installed): iso-builder/live-build/config/includes.chroot/etc/systemd/system/synvirt-{updaterd,migrationd}.service.

Docs: docs/daemon-split/{DESIGN,PROGRESS,REPORT}.md.

Config keys added (with defaults)

Section titled “Config keys added (with defaults)”

Daemon (DaemonConfig, client side) — both overridable; None uses the product-convention path:

  • updater_socket_path → /run/synvirt/updaterd.sock
  • migration_socket_path → /run/synvirt/migrationd.sock

synvirt-updaterd (/etc/synvirt/updaterd.toml, env SYNVIRT_UPDATERD_*):

  • socket_path → /run/synvirt/updaterd.sock (env SYNVIRT_UPDATERD_SOCKET)
  • socket_mode → 0o600
  • boot_env_pool → synvirt-root

synvirt-migrationd (/etc/synvirt/migrationd.toml, env SYNVIRT_MIGRATIOND_*):

  • socket_path → /run/synvirt/migrationd.sock (env SYNVIRT_MIGRATIOND_SOCKET)
  • socket_mode → 0o600
  • db_path → /var/lib/synvirt/migrationd.db (env SYNVIRT_MIGRATIOND_DB)

OpenAPI impact

Section titled “OpenAPI impact”

Verified with synvirt-daemon --dump-openapi (before = pre-split HEAD, after = this branch):

  • Update routes: byte-identical except one operation description line — the logs SSE endpoint now references synvirt_updaterd::ProgressEvent instead of the deleted crate::update_v2::ProgressEvent. No path, method, operation_id, parameter, or schema changed.
  • Migration routes: purely additive — +3 paths (/api/v1/system/migration/jobs, …/{id}, …/{id}/cancel), +4 schemas (Job, JobKind, JobState, CreateJobRequest).
  • The committed crates/web-ux-v2/openapi.snapshot.json was already stale vs HEAD before this work (59-line pre-existing drift) and was left untouched. Regenerating it is a separate frontend chore (see open questions).

Morning deploy checklist (WRITTEN ONLY — NOT executed)

Section titled “Morning deploy checklist (WRITTEN ONLY — NOT executed)”

Nothing below was run tonight. No host was contacted. Review before applying. Standard caution: the updater can stop/restart the daemon, so stage it when you can watch the box.

  1. Build the workspace release binaries on the dev box: cargo build --release -p synvirt-updaterd -p synvirt-migrationd -p synvirt-daemon (the daemon must be rebuilt — it now proxies instead of running the orchestrator in-process).
  2. Stage binaries to each target host at /opt/synvirt/bin/: synvirt-updaterd, synvirt-migrationd, and the rebuilt synvirt-daemon.
  3. Install the units (from the repo packaging dir) into /etc/systemd/system/: synvirt-updaterd.service, synvirt-migrationd.service. systemctl daemon-reload.
  4. Ensure runtime dirs: /run/synvirt (the units’ ExecStartPre already install -d these) and /var/lib/synvirt.
  5. Start order (sub-daemons first, then the API daemon so its first proxied request finds a live socket): systemctl enable --now synvirt-updaterd.service → systemctl enable --now synvirt-migrationd.service → systemctl restart synvirt-daemon.service. (If a sub-daemon is down, the daemon returns 503 E_UPDATER_UNAVAILABLE / E_MIGRATION_UNAVAILABLE — no panic — so ordering is not critical for safety, only for first-request success.)
  6. Verify:
    • systemctl status synvirt-updaterd synvirt-migrationd
    • curl --unix-socket /run/synvirt/updaterd.sock http://localhost/healthz
    • curl --unix-socket /run/synvirt/migrationd.sock http://localhost/healthz
    • Through the daemon (authenticated): GET /api/v1/system/update/status → idle; GET /api/v1/system/migration/jobs → [].
  7. Rollback: systemctl disable --now synvirt-updaterd synvirt-migrationd, restore the previous synvirt-daemon binary, systemctl restart synvirt-daemon. (The pre-split daemon ran the orchestrator in-process, so the previous binary is self-sufficient.)

Lifecycle invariants baked into the units:

  • synvirt-updaterd.service has no ordering/binding relationship with synvirt-daemon — a control-plane restart never touches it, and it can stop/restart the daemon during an apply.
  • synvirt-migrationd.service is After=libvirtd.service ordering-only (no Requires/PartOf/Conflicts), independent of both synvirt-daemon and synvirt-guest-drain.

Blockers & open questions

Section titled “Blockers & open questions”
  1. Real libvirt migration executor is a documented stub. Per the no-real-migration rule, LibvirtExecutor::execute fails fast with “not implemented”. A created migration job runs queued→preparing→running→failed. Next step: implement virDomainMigrate* (live/offline) between source and target hosts; pin the unit’s capability set then.
  2. synvirt-updaterd version string drops the +build.N git-counter suffix the in-process daemon used for current_version (semver comparisons ignore build metadata, so update logic is unaffected — only the cosmetic string differs). Add a build-counter build.rs if exact parity is wanted.
  3. OpenAPI snapshot (crates/web-ux-v2/openapi.snapshot.json) is stale vs HEAD (pre-existing) and now also lacks the 4 additive migration schemas + the one update/logs description tweak. Regenerate with synvirt-daemon --dump-openapi > openapi.snapshot.json and npm run generate:api:from-file when doing frontend work for migration.
  4. No frontend for the migration surface yet (backend + OpenAPI only).
  5. Sub-daemon hardening: synvirt-updaterd runs as root with CAP_SYS_ADMIN (its apply phase shells out to mount/systemctl reboot). Tightening to a minimal privilege model is a follow-up.
  6. Enable flags (updater.enabled / migration.enabled from the design) were not wired — the proxy routes are always mounted (so the OpenAPI surface is static) and a down sub-daemon yields 503. Add the flags if conditional mounting is desired.
  7. Environment: see the three env caveats above (guest-installer payloads, live fonts, webrtc SVT-AV1 flake). None block the split.