Design — extract the migrator import engine into synvirt-migratord

Synced read-only from /home/synnet/mirrors/synvirt-product/docs/daemon-split/DESIGN-migratord.md. Edit at the source, not here.

Design — extract the migrator import engine into synvirt-migratord

Status: design (feature/daemon-split). 2026-06-10.

Goal: move the existing synvirt-migrator import engine (JobManager, sled StateStore, source/upload registries, the orchestrator pipeline, the go-shim, all daemon-side wiring) out of synvirt-daemon into a new binary crate synvirt-migratord on the synvirt-ipc transport, so a synvirt-daemon restart never kills an in-flight import. synvirt-daemon proxies /api/v1/migrator/* 1:1 over a UDS. synvirt-migrationd (the greenfield vMotion stub) is untouched and out of scope.

0. The coupling that shapes the design

Unlike the updater orchestrator (self-contained), the migrator is not self-contained. The JobManager holds two injected traits whose only real implementations live INSIDE the daemon and reach into the daemon’s VM-creation brain:

• migrator::orchestrator::DomainStarter — implemented by daemon::migrator_domain::ProfileDomainStarter, which calls crate::vm_profiles::resolve, crate::libvirt_renderer::render_domain_xml, crate::telemetry::CidMap, crate::cpu, and synvirt_core::storage::vm_layout. Methods: define_and_start(manifest, disks, nic_mappings, disk_mappings, pool, auto_start, boot_mode) -> Uuid, connect_nics(domain), destroy_and_undefine(domain). All inputs are serde-able migrator types (DestinationDisk needs a derive added — internal type, not wire).
• migrator::orchestrator::PoolResolver — implemented by daemon::migrator_domain::RegistryPoolResolver over the daemon’s synvirt_storage pool registry. Method: resolve_zpool(&str) -> String (sync, infallible).

Moving the JobManager therefore requires a seam for these two traits. Extracting vm_profiles/libvirt_renderer/telemetry/cpu into shared crates would be a forbidden core redesign. So the faithful, minimal seam is a reverse UDS callback: the daemon keeps migrator_domain unchanged and serves it on a private socket; migratord implements the two traits as thin UDS clients. The real domain-definition code never moves, so the migrated-domain behavior is byte-for-byte what it is today; only the trait invocation crosses a socket.

This is two UDS channels:

• daemon → migratord (migrator.socket_path, default /run/synvirt/migratord.sock): the public-route reverse proxy.
• migratord → daemon (migrator.callback_socket_path, default /run/synvirt/migratord-callback.sock): the DomainStarter / PoolResolver callbacks.

1. Crate layout

crates/synvirt-migratord/   # NEW lib+bin
  src/main.rs               # construct StateStore/sources/uploads/JobManager, serve router over UDS
  src/config.rs             # MigratordConfig (socket_path, callback_socket_path, db/keyring/uploads/import roots)
  src/remote_starter.rs     # RemoteDomainStarter (impl DomainStarter over UDS, async)
  src/remote_resolver.rs    # RemotePoolResolver (impl PoolResolver over UDS, sync/blocking)
  src/callback_proto.rs     # shared DTOs for the callback wire (also used by the daemon server)

The callback DTOs live in migratord’s lib and the daemon depends on it for the server side (mirrors how the daemon depends on synvirt-updaterd for its wire types). synvirt-migrator (the engine crate) is unchanged except adding serde to DestinationDisk.

2. What moves vs. what stays

Moves to migratord (today in daemon/src/main.rs:1174-1309): Cipher, StateStore::open (the sled at /var/lib/synvirt/migrator/state.db), SourceRegistry, UploadStore, VirtV2vInPlacePreparer self-check + preparer_available, InventoryStore, JobManager::new(...).with_import_root().with_pool_resolver(), the AppState, and the synvirt_migrator::router(...) serving.

Stays in the daemon: migrator_domain::{ProfileDomainStarter, RegistryPoolResolver} (the real pipeline) + mig_network, cid_map, mig_storage_registry they need — now wired into a small callback server instead of into a JobManager. The observability bridge stays daemon-side but is re-sourced from migratord’s stream (see §5).

Sled single ownership (rule 6): after the move, StateStore::open and the literal /var/lib/synvirt/migrator path appear ONLY in migratord. Grep-gate before commit: grep -rn "StateStore::open\|/var/lib/synvirt/migrator" crates/daemon/src → 0.

3. Public proxy (wire-frozen, rule 5)

/api/v1/migrator/* is absent from the OpenAPI today (the router is mounted but never registered in the #[openapi] derive). So the wire-frozen gate is simply: synvirt-daemon --dump-openapi is byte-identical before/after (178 paths, no migrator entries appear). The proxy handlers carry no #[utoipa::path] annotations (matching today’s absence). Route table proxied 1:1 (from migrator::http::router): /sources*, /probe-cert, /plans*, /networks/{plan,validate}, /migrations* (incl. /migrations/stream SSE — streamed, not buffered), /uploads*, /host-files*, /capabilities.

A wildcard proxy route (/api/v1/migrator/*rest) under the existing PAM Basic-Auth layer forwards method + path + query + body to the migratord socket and streams the response. Socket down → 503 E_MIGRATOR_UNAVAILABLE in the standard envelope, never a panic. synvirt_ipc::UdsClient::forward already streams the response body (verified in the updaterd phase), so SSE works through the proxy.

4. Reverse callback contract (migratord → daemon)

Daemon serves (on the callback socket) a tiny axum router:

• POST /define-and-start ← {manifest, disks, nic_mappings, disk_mappings, pool, auto_start, boot_mode} → {uuid} | error
• POST /connect-nics ← {domain} → {} | error
• POST /destroy-and-undefine ← {domain} → {} | error
• POST /resolve-zpool ← {label} → {zpool} (infallible)

It calls the kept ProfileDomainStarter / RegistryPoolResolver verbatim. migratord’s RemoteDomainStarter (async) and RemotePoolResolver (sync — a blocking std-UnixStream HTTP/1.1 call, since resolve_zpool is sync and called once per job) are the clients. Errors map back to migrator::Error so job failure semantics are unchanged. If the callback socket is down at define time the job fails with a clear error (same shape as a today-internal failure) — it never hangs; the daemon is normally up because define happens at the end of a live operator-driven import.

5. Tasks/Events preservation (the silent-breakage zone)

Today daemon/src/main.rs:1257 subscribes to the in-process mig_jobs.subscribe() broadcast and, on each job’s first Failed, publishes a task.migration.failed event into observability (obs.bus/obs.db) → dashboard Activity + WS. After the split the JobManager is in migratord. Preservation: the daemon runs a background task that tails migratord’s /migrations/stream SSE (over the proxy socket), detects first-time Failed transitions, and publishes the same task_failed_event into its own observability bus (which stays daemon-side, sled-owned by the daemon). Same event, same code/message, same dashboard surface — only the source of the signal moves from an in-process channel to a UDS stream.

6. go-shim (govmomi)

The vim_api adapter links the Go shim as a C archive at compile time (migrator/build.rs → libvim_shim.a, cargo:rustc-cfg=vim_shim, FFI in adapters/vim_api/ffi.rs); absent Go toolchain → Rust stub fallback. There is no separate runtime process or artifact to package — the shim is statically linked into whatever binary hosts the migrator crate, now synvirt-migratord. The release build must run on a box with Go ≥1.21 so the shim links (the dev box has it). No new path config key is needed.

7. systemd unit (repo only, NOT installed)

synvirt-migratord.service: mirrors synvirt-daemon’s privileges (it defines domains via the callback and creates zvols / runs zfs + adapter I/O directly): runs as root, After=local-fs.target libvirtd.service ordering-only (no Requires/PartOf/BindsTo on daemon or libvirtd, no Conflicts with guest-drain), RuntimeDirectory=synvirt, ReadWritePaths=/etc/synvirt /var/lib/synvirt /run/synvirt, Restart=on-failure. Config key migrator.socket_path default /run/synvirt/migratord.sock.

8. Startup / in-flight state

JobManager does not auto-resume in-flight sled jobs on construction (confirmed: no resume/recover path; an orphaned copying_full job on .75 is exactly this — abandoned across a past daemon restart, not resumed). migratord reproduces this identically: it opens the StateStore and serves; in-flight jobs from a prior process are left in their last sled state (visible in the list, not resurrected). The split’s win is the opposite direction: an import running in the migratord process is unaffected by a synvirt-daemon restart (the daemon is just the proxy), which is the “money shot” Tony validates manually.

9. Deploy & SEV-0 (see REPORT for execution)

Per-host swap order honors the sled lock: stop synvirt-daemon (Fix70 → guests untouched; if the host’s running daemon predates Fix70 / lacks guest-drain, stop via SIGKILL so its shutdown handler never drains guests) → start migratord (verify it acquired the StateStore + answers health) → start updaterd → start the new daemon. PID snapshot before/after every stop/start; any pre-existing QEMU PID changed → roll back that host. .75 is FROZEN (non-terminal copying_full job — Phase 0 gate); deploy order this session is .56 → .65.

10. Risks

• The define path runs unchanged daemon-side code but its inputs now cross a socket as JSON; a serde-fidelity bug would only surface in a real import (Tony’s morning test) — mitigated by serde round-trip unit tests on VmManifest/DestinationDisk/NicMapping/DiskMapping/BootMode and by deploying .56 (no guests) first. A define bug fails the test import + rolls back; it cannot harm an existing guest (define creates a new domain).
• Two sockets increase moving parts; both fail closed (503 / job error).