virt/hyperv/vmbus.rst

1 .. SPDX-License-Identifier: GPL-2.0
5 VMBus is a software construct provided by Hyper-V to guest VMs.  It
7 devices that Hyper-V presents to guest VMs.   The control path is
8 used to offer synthetic devices to the guest VM and, in some cases,
10 channels for communicating between the device driver in the guest VM
11 and the synthetic device implementation that is part of Hyper-V, and
12 signaling primitives to allow Hyper-V and the guest to interrupt
16 entry in a running Linux guest.  The VMBus driver (drivers/hv/vmbus_drv.c)
17 establishes the VMBus control path with the Hyper-V host, then
21 Most synthetic devices offered by Hyper-V have a corresponding Linux
29 * PCI device pass-thru
34 * Key/Value Pair (KVP) exchange with Hyper-V
35 * Hyper-V online backup (a.k.a. VSS)
37 Guest VMs may have multiple instances of the synthetic SCSI
38 controller, synthetic NIC, and PCI pass-thru devices.  Other
41 Hyper-V that are used only by Windows guests and for which Linux
44 Hyper-V uses the terms "VSP" and "VSC" in describing synthetic
45 devices.  "VSP" refers to the Hyper-V code that implements a
47 the device in the guest VM.  For example, the Linux driver for the
53 --------------
55 between the VSP and the VSC.  Channels are bi-directional and used
64 The "in" ring buffer is for messages from the Hyper-V host to the
65 guest, and the "out" ring buffer is for messages from the guest to
66 the Hyper-V host.  In Linux, the "in" and "out" designations are as
67 viewed by the guest side.  The ring buffers are memory that is
68 shared between the guest and the host, and they follow the standard
69 paradigm where the memory is allocated by the guest, with the list
74 guest and is specific to each synthetic device.   The list of GPAs
75 making up the ring is communicated to the Hyper-V host over the
84 ring buffer need not be concerned with ring buffer wrap-around.
89 directly in the ring without handling wrap-around.
92 passed to Hyper-V as a 4 Kbyte area.  But the memory for the actual
96 Hyper-V.  This case is handled by vmbus_establish_gpadl().
98 Hyper-V enforces a limit on the aggregate amount of guest memory
100 that a rogue guest can't force the consumption of excessive host
106 ----------------------
114 * Unidirectional:  Either side sends a message and does not
116 * Request/response:  One side (usually the guest) sends a message
120 responses.  Some synthetic devices allow multiple requests to be in-
121 flight simultaneously, so the guest specifies a transactionID when
122 sending a request.  Hyper-V sends back the same transactionID in the
128 between the guest and the Hyper-V host, the actual data to be
130 specified as a separate data buffer that the Hyper-V host will
134 Hyper-V host to the guest contain the actual time value.  When the
142 1. vmbus_sendpacket():  Control-only messages and messages with
143    embedded data -- no GPAs
147    of guest memory can be targeted.
151    single logical area of guest memory to be targeted.
153 Historically, Linux guests have trusted Hyper-V to send well-formed
156 technologies that fully encrypt guest memory and that allow the
157 guest to not trust the hypervisor (AMD SEV-SNP, Intel TDX), trusting
158 the Hyper-V host is no longer a valid assumption.  The drivers for
160 values read from memory that is shared with Hyper-V, which includes
162 messages read by the guest from the "in" ring buffer are copied to a
163 temporary buffer that is not shared with Hyper-V.  Validation is
164 performed in this temporary buffer without the risk of Hyper-V
169 --------------------------------------
170 Hyper-V provides each guest CPU with a synthetic interrupt controller
171 that is used by VMBus for host-guest communication. While each synic
174 the Hyper-V host and a guest CPU use that SINT.
176 The SINT is mapped to a single per-CPU architectural interrupt (i.e,
177 an 8-bit x86/x64 interrupt vector, or an arm64 PPI INTID). Because
178 each CPU in the guest has a synic and may receive VMBus interrupts,
179 they are best modeled in Linux as per-CPU interrupts. This model works
180 well on arm64 where a single per-CPU Linux IRQ is allocated for
182 "Hyper-V VMbus". Since x86/x64 lacks support for per-CPU IRQs, an x86
200 ----------------
201 VMBus provides a mechanism for the guest to interrupt the host when
202 the guest has queued new messages in a ring buffer.  The host
203 expects that the guest will send an interrupt only when an "out"
204 ring buffer transitions from empty to non-empty.  If the guest sends
206 unnecessary.  If a guest sends an excessive number of unnecessary
207 interrupts, the host may throttle that guest by suspending its
208 execution for a few seconds to prevent a denial-of-service attack.
210 Similarly, the host will interrupt the guest via the synic when
212 channel "in" ring buffer transitions from empty to non-empty due to
223 The guest CPU that a VMBus channel will interrupt is selected by the
224 guest when the channel is created, and the host is informed of that
247 When running on later versions of Hyper-V, the CPU can be changed
252 An online CPU in a Linux guest may not be taken offline if it has
261 CPU-based exclusion for correctness.  In normal operation, Hyper-V
263 channel is being changed via sysfs, the guest doesn't know exactly
264 when Hyper-V will make the transition.  The code must work correctly
265 even if there is a time lag before Hyper-V starts interrupting the
269 ------------------------------
270 Hyper-V and the Linux guest have a separate message-passing path
275 The first step is for the guest to connect to the generic
276 Hyper-V VMBus mechanism.  As part of establishing this connection,
277 the guest and Hyper-V agree on a VMBus protocol version they will
279 Hyper-V versions, and vice versa.
281 The guest then tells Hyper-V to "send offers".  Hyper-V sends an
282 offer message to the guest for each synthetic device that the VM
285 identified by a GUID. The offer message from Hyper-V contains
289 class ID. The ordering of offer messages can vary from boot-to-boot
292 because Hyper-V supports adding devices, such as synthetic NICs,
296 Upon receipt of an offer message, the guest identifies the device
302 the corresponding VSP. It allocates guest memory for the channel
303 ring buffers and shares the ring buffer with the Hyper-V host by
310 VSC and the VSP on the Hyper-V host.  The setup messages may also
312 mis-named as "sub-channels" since they are functionally
318 The Hyper-V host can send a "rescind" message to the guest to
323 rescinded, neither Hyper-V nor Linux retains any state about
324 its previous existence. Such a device might be re-added later,