qemu/qapi
Steve Sistare 624e6e654e migration: cpr-transfer mode
Add the cpr-transfer migration mode, which allows the user to transfer
a guest to a new QEMU instance on the same host with minimal guest pause
time, by preserving guest RAM in place, albeit with new virtual addresses
in new QEMU, and by preserving device file descriptors.  Pages that were
locked in memory for DMA in old QEMU remain locked in new QEMU, because the
descriptor of the device that locked them remains open.

cpr-transfer preserves memory and devices descriptors by sending them to
new QEMU over a unix domain socket using SCM_RIGHTS.  Such CPR state cannot
be sent over the normal migration channel, because devices and backends
are created prior to reading the channel, so this mode sends CPR state
over a second "cpr" migration channel.  New QEMU reads the cpr channel
prior to creating devices or backends.  The user specifies the cpr channel
in the channel arguments on the outgoing side, and in a second -incoming
command-line parameter on the incoming side.

The user must start old QEMU with the the '-machine aux-ram-share=on' option,
which allows anonymous memory to be transferred in place to the new process
by transferring a memory descriptor for each ram block.  Memory-backend
objects must have the share=on attribute, but memory-backend-epc is not
supported.

The user starts new QEMU on the same host as old QEMU, with command-line
arguments to create the same machine, plus the -incoming option for the
main migration channel, like normal live migration.  In addition, the user
adds a second -incoming option with channel type "cpr".  This CPR channel
must support file descriptor transfer with SCM_RIGHTS, i.e. it must be a
UNIX domain socket.

To initiate CPR, the user issues a migrate command to old QEMU, adding
a second migration channel of type "cpr" in the channels argument.
Old QEMU stops the VM, saves state to the migration channels, and enters
the postmigrate state.  New QEMU mmap's memory descriptors, and execution
resumes.

The implementation splits qmp_migrate into start and finish functions.
Start sends CPR state to new QEMU, which responds by closing the CPR
channel.  Old QEMU detects the HUP then calls finish, which connects the
main migration channel.

In summary, the usage is:

  qemu-system-$arch -machine aux-ram-share=on ...

  start new QEMU with "-incoming <main-uri> -incoming <cpr-channel>"

  Issue commands to old QEMU:
    migrate_set_parameter mode cpr-transfer

    {"execute": "migrate", ...
        {"channel-type": "main"...}, {"channel-type": "cpr"...} ... }

Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Link: https://lore.kernel.org/r/1736967650-129648-17-git-send-email-steven.sistare@oracle.com
Signed-off-by: Fabiano Rosas <farosas@suse.de>
2025-01-29 11:56:24 -03:00
..
acpi.json qapi: convert "Example" sections without titles 2024-07-17 10:20:53 +02:00
audio.json audio/pw: Pipewire->PipeWire case fix for user-visible text 2023-07-17 15:22:56 +04:00
authz.json qapi: Reformat doc comments to conform to current conventions 2023-05-10 10:01:01 +02:00
block-core.json block-backend: per-device throttling of BLOCK_IO_ERROR reports 2024-10-22 17:52:49 +02:00
block-export.json nbd/server: CVE-2024-7409: Cap default max-connections to 100 2024-08-08 16:02:23 -05:00
block.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
char.json chardev: add path option for pty backend 2024-10-09 12:13:05 +04:00
common.json qapi/common: Supply missing member documentation 2024-10-02 09:10:01 +02:00
compat.json qapi: Belatedly update CompatPolicy documentation for unstable 2023-10-19 07:02:29 +02:00
control.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
crypto.json crypto: Introduce SM3 hash hmac pbkdf algorithm 2024-11-05 18:37:18 +00:00
cryptodev.json qapi: Document QCryptodevBackendServiceType 2024-10-02 09:10:01 +02:00
cxl.json qapi/cxl: Supply missing member documentation 2024-10-02 09:10:01 +02:00
dump.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
ebpf.json qapi/ebpf: Drop temporary 'prefix' 2024-09-10 13:22:47 +02:00
error.json qapi: Reformat doc comments to conform to current conventions 2023-05-10 10:01:01 +02:00
introspect.json qapi/introspect: Supply missing member documentation 2024-10-02 09:10:01 +02:00
job.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
machine-common.json qapi/qom: Define cache enumeration and properties for machine 2024-11-05 23:32:25 +00:00
machine-target.json qapi/machine: Rename CpuS390* to S390Cpu*, and drop 'prefix' 2024-09-10 13:22:47 +02:00
machine.json Revert "Remove the unused sh4eb target" 2024-11-04 14:16:11 +01:00
meson.build qapi/vfio: Add VFIO migration QAPI event 2024-05-16 16:59:19 +02:00
migration.json migration: cpr-transfer mode 2025-01-29 11:56:24 -03:00
misc-target.json qapi: convert "Example" sections without titles 2024-07-17 10:20:53 +02:00
misc.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
net.json net/stream: deprecate 'reconnect' in favor of 'reconnect-ms' 2024-10-28 14:37:25 +08:00
opts-visitor.c qapi: Inline and remove QERR_INVALID_PARAMETER definition 2024-04-24 09:50:58 +02:00
pci.json qapi/pci: Supply missing member documentation 2024-10-02 09:10:01 +02:00
pragma.json qapi: Document QCryptodevBackendServiceType 2024-10-02 09:10:01 +02:00
qapi-clone-visitor.c qapi: Do not cast function pointers 2024-05-29 12:41:56 +02:00
qapi-dealloc-visitor.c qapi: Make visitor functions taking Error ** return bool, not void 2020-07-10 15:18:08 +02:00
qapi-forward-visitor.c qapi: remove needless include 2022-03-22 14:46:18 +04:00
qapi-schema.json qapi/vfio: Add VFIO migration QAPI event 2024-05-16 16:59:19 +02:00
qapi-type-helpers.c qapi: New strv_from_str_list() 2024-03-04 07:12:40 +01:00
qapi-util.c qapi: Fix dangling references to docs/devel/qapi-code-gen.txt 2024-01-26 07:04:53 +01:00
qapi-visit-core.c qapi: Factor out compat_policy_input_ok() 2021-10-29 21:27:20 +02:00
qdev.json qapi: fix device-sync-config since-version 2024-11-26 17:18:06 -05:00
qmp-dispatch.c Revert "monitor: use aio_co_reschedule_self()" 2024-06-10 11:05:43 +02:00
qmp-event.c Replace qemu_gettimeofday() with g_get_real_time() 2022-04-06 10:50:37 +02:00
qmp-registry.c qapi: Generalize command policy checking 2021-10-29 18:24:46 +02:00
qobject-input-visitor.c qapi: Inline QERR_INVALID_PARAMETER_TYPE definition (constant value) 2024-04-24 09:50:58 +02:00
qobject-output-visitor.c qapi: Extend -compat to set policy for unstable interfaces 2021-10-29 21:28:01 +02:00
qom.json qapi/qom: Change Since entry for AcpiGenericPortProperties to 9.2 2024-11-26 17:18:06 -05:00
replay.json qapi: convert "Example" sections without titles 2024-07-17 10:20:53 +02:00
rocker.json qapi/rocker: Supply missing member documentation 2024-10-02 09:10:01 +02:00
run-state.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
sockets.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
stats.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
string-input-visitor.c qapi: Inline QERR_INVALID_PARAMETER_TYPE definition (constant value) 2024-04-24 09:50:58 +02:00
string-output-visitor.c string-output-visitor: Fix (pseudo) struct handling 2024-01-26 11:16:58 +01:00
tpm.json qapi: convert "Example" sections without titles 2024-07-17 10:20:53 +02:00
trace-events qapi: Generalize struct member policy checking 2021-10-29 18:23:09 +02:00
trace.h trace: switch position of headers to what Meson requires 2020-08-21 06:18:24 -04:00
trace.json qapi: convert "Example" sections without titles 2024-07-17 10:20:53 +02:00
transaction.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
ui.json qapi/ui: Drop temporary 'prefix' 2024-09-10 13:22:47 +02:00
vfio.json qapi/vfio: Rename VfioMigrationState to Qapi*, and drop prefix 2024-09-10 14:03:45 +02:00
virtio.json qapi: Refill doc comments to conform to conventions 2024-08-05 09:31:51 +02:00
yank.json qapi: convert "Example" sections without titles 2024-07-17 10:20:53 +02:00