feat(vmm): add netd-managed macvtap networking - #1061
Open
kvinwang wants to merge 10 commits into
Open
Conversation
kvinwang
force-pushed
the
feat/vmm-macvtap-netd
branch
from
August 17, 2026 08:44
d17c83a to
af3bf8d
Compare
kvinwang
force-pushed
the
feat/vmm-macvtap-netd
branch
from
August 17, 2026 12:52
af3bf8d to
c943fa6
Compare
kvinwang
marked this pull request as ready for review
August 17, 2026 13:01
Contributor
There was a problem hiding this comment.
Pull request overview
Adds first-class macvtap networking support to dstack-vmm, extending the VMM ↔ netd contract to prepare macvtap interfaces and pass the resulting runtime /dev/tapN device nodes to QEMU via pre-opened FDs (instead of persisting device paths in manifests). This fits into the VMM’s networking and process-launch pipeline by expanding the networking model/config validation, RPC surface, netd operations, and the per-VM launcher spec.
Changes:
- Add macvtap networking mode end-to-end (manifest config model, VMM RPC proto, netd prepare op, QEMU arg building).
- Extend the per-VM launcher spec with
open_filesand optionalswtpm/socket fields to support passing pre-opened device FDs. - Remove the unused “run QEMU as another user via sudo” setting and update docs/config examples accordingly.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| dstack/vmm/vmm.toml | Removes unused user config field from VMM config template. |
| dstack/vmm/src/vm_launcher.rs | Adds open_files FD mapping support and makes swtpm optional in the launcher spec. |
| dstack/vmm/src/netd.rs | Adds prepare_macvtap netd operation, returns runtime device node, and adjusts cleanup semantics. |
| dstack/vmm/src/main_service.rs | Accepts macvtap networking mode via RPC and advertises it in metadata. |
| dstack/vmm/src/config.rs | Adds macvtap fields to Networking, validates macvtap config, removes cvm.user. |
| dstack/vmm/src/app/vm_info.rs | Exposes macvtap mode fields in VM info protobuf mapping. |
| dstack/vmm/src/app/qemu.rs | Builds QEMU -netdev tap,fd=... for macvtap and wraps launch via launcher when needed. |
| dstack/vmm/src/app.rs | Integrates netd macvtap preparation into VM start/stop lifecycle and rollback flow. |
| dstack/vmm/rpc/proto/vmm_rpc.proto | Extends NetworkingConfig with parent and macvtap_mode. |
| dstack/vmm/Cargo.toml | Enables additional nix features used by the new launcher logic. |
| docs/tutorials/vmm-configuration.md | Updates config example to remove user. |
| docs/macvtap-networking.md | Adds new documentation for macvtap networking configuration and lifecycle. |
Suppressed comments (1)
dstack/vmm/src/netd.rs:404
remove_interfacenow also deletes macvtap interfaces, but the log message still says "removed filtered TAP", which is misleading for macvtap (no libvirt nwfilter binding).
if Path::new("/sys/class/net").join(tap).exists() {
ip(&["link", "delete", "dev", tap])?;
info!(%tap, "removed filtered TAP");
}
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+583
to
587
| let response = netd::request(&self.config.netd.socket, &request).await; | ||
| if let Err(error) = response { | ||
| // The client may have timed out while netd was still finishing | ||
| // this Prepare. Remove the in-flight identity first; netd's | ||
| // serialized accept loop processes it after Prepare completes. |
Comment on lines
+478
to
+483
| if let Err(error) = work_dir.set_runtime_networks(&runtime_networks) { | ||
| let _ = self | ||
| .remove_filtered_networks(&vm_config.manifest.id, &runtime_networks) | ||
| .await; | ||
| return Err(error); | ||
| } |
Comment on lines
+36
to
+40
| The per-VM launcher opens the character device, places it at the fd referenced | ||
| by QEMU's `-netdev tap,fd=...` argument, and then execs QEMU. This keeps device | ||
| paths out of persistent VM | ||
| configuration, works with both Supervisor and systemd process managers, and | ||
| does not pass network fds through `sudo`. |
Comment on lines
+258
to
+263
| let mut qemu = match spawn_child(&spec.qemu, &spec.open_files) { | ||
| Ok(child) => child, | ||
| Err(error) => { | ||
| stop_child(&mut swtpm, "swtpm", grace).await; | ||
| if let Some(child) = &mut swtpm { | ||
| stop_child(child, "swtpm", grace).await; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is an alternative to #1047. Device paths are runtime results from netd rather than persistent manifest inputs, so interface identity and lifecycle stay inside dstack and ifindex reuse cannot silently redirect a VM to a stale path.
The existing bridge prepare RPC is renamed independently in #1064.
Design
A macvtap NIC is configured with a parent interface and forwarding mode. The VMM and QEMU use the same deterministic guest MAC. Netd derives the stable interface name, replaces stale state, creates the macvtap, reads its ifindex, waits for the character device, and returns the path for this launch only.
The launcher opens each device, reserves collision-free source fds, and maps them to the QEMU target fds. For a single-process launch it execs QEMU in place; with swtpm it keeps the parent and child orchestration path.
Validation
Not yet tested