Block layer patches:

- qemu-storage-daemon: add --pidfile option
 - qemu-storage-daemon: CLI error messages include the option name now
 - vhost-user-blk export: Misc fixes
 - docs: Improvements for qemu-storage-daemon documentation
 - parallels: load bitmap extension
 - backup-top: Don't crash on post-finalize accesses
 - Improve error messages related to node-name options
 - iotests improvements
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmBGWHURHGt3b2xmQHJl
 ZGhhdC5jb20ACgkQfwmycsiPL9ZpyxAAk0gRiayMUidSzgvzU/CeUhzBsC4ayEkn
 dLtTZ8hl7cW/w3GjDK1Wri4MANRN/0YHjiLSzO38lfVpK0z8SJr5aU4CwhRlOKVm
 VWgx+OLlV4Azht9fMNF4SwUXgXhl7pUNiFMNnomb++gvqhjMCedDZcWlnVKhbuQ+
 O3TKGO4tToSGaXP85jCM4xukw5HZ//4QMYg6MH0gDk8ahfE2MhyTHz64oDp412os
 qhxvc4bU2S5xGLaBfLGhsc6VPQFKjblG704P/Y73zeoxq12A0L2Ru98WvrNaXw7Z
 m54jJUINiDkJ7ZOl6W04zdeiLvs3BOrNe+7mxawOTmdkBsLOKErrhrTO1gJmHHmX
 kJLWEh9VYWxVbvE7C3KQt9bclR6wt+aPup4X1XE8pHtocPVONVq5bvctrVgxgK0b
 btN06NcK+2jQxcQkG4MnBJ8S41qmxHyIEQlQWKyUWXvKt6zsFU/NuWKMQrAfYZZi
 5J+RPU/fB073LY4lpAgou0OP1/RIvQmi5zWzjWm/Qbp3JpgC+azcYvxn7UU7J71P
 +u8IEQ4+Q9s0gvXQAh/U8AQg2eOqAwEAyFUJl9wpPN56O03dbI8KyCV5ECIRJu49
 CC8uKlJxZkbw9ZBs11SAmm/0J64WcNb2AMWxDPC8Z6oQbVaRRRznoRwRP2H6odUu
 uBolS43+5cI=
 =eAjH
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qemu-storage-daemon: add --pidfile option
- qemu-storage-daemon: CLI error messages include the option name now
- vhost-user-blk export: Misc fixes
- docs: Improvements for qemu-storage-daemon documentation
- parallels: load bitmap extension
- backup-top: Don't crash on post-finalize accesses
- Improve error messages related to node-name options
- iotests improvements

# gpg: Signature made Mon 08 Mar 2021 17:01:41 GMT
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (30 commits)
  blockdev: Clarify error messages pertaining to 'node-name'
  block: Clarify error messages pertaining to 'node-name'
  docs: qsd: Explain --export nbd,name=... default
  MAINTAINERS: update parallels block driver
  iotests: add parallels-read-bitmap test
  iotests.py: add unarchive_sample_image() helper
  parallels: support bitmap extension for read-only mode
  block/parallels: BDRVParallelsState: add cluster_size field
  parallels.txt: fix bitmap L1 table description
  qcow2-bitmap: make bytes_covered_by_bitmap_cluster() public
  block/export: port virtio-blk read/write range check
  block/export: port virtio-blk discard/write zeroes input validation
  block/export: fix vhost-user-blk export sector number calculation
  block/export: use VIRTIO_BLK_SECTOR_BITS
  block/export: fix blk_size double byteswap
  libqtest: add qtest_remove_abrt_handler()
  libqtest: add qtest_kill_qemu()
  libqtest: add qtest_socket_server()
  vhost-user-blk: fix blkcfg->num_queues endianness
  docs: replace insecure /tmp examples in qsd docs
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
This commit is contained in:
Peter Maydell 2021-03-09 21:31:18 +00:00
commit a557b00469
51 changed files with 969 additions and 175 deletions

View file

@ -208,21 +208,25 @@ of its data area are:
28 - 31: l1_size
The number of entries in the L1 table of the bitmap.
variable: l1_table (8 * l1_size bytes)
L1 offset table (in bytes)
variable: L1 offset table (l1_table), size: 8 * l1_size bytes
A dirty bitmap is stored using a one-level structure for the mapping to host
clusters - an L1 table.
The dirty bitmap described by this feature extension is stored in a set of
clusters inside the Parallels image file. The offsets of these clusters are
saved in the L1 offset table specified by the feature extension. Each L1 table
entry is a 64 bit integer as described below:
Given an offset in bytes into the bitmap data, the offset in bytes into the
image file can be obtained as follows:
Given an offset in bytes into the bitmap data, corresponding L1 entry is
offset = l1_table[offset / cluster_size] + (offset % cluster_size)
l1_table[offset / cluster_size]
If an L1 table entry is 0, the corresponding cluster of the bitmap is assumed
to be zero.
If an L1 table entry is 0, all bits in the corresponding cluster of the bitmap
are assumed to be 0.
If an L1 table entry is 1, the corresponding cluster of the bitmap is assumed
to have all bits set.
If an L1 table entry is 1, all bits in the corresponding cluster of the bitmap
are assumed to be 1.
If an L1 table entry is not 0 or 1, it allocates a cluster from the data area.
If an L1 table entry is not 0 or 1, it contains the corresponding cluster
offset (in 512b sectors). Given an offset in bytes into the bitmap data the
offset in bytes into the image file can be obtained as follows:
offset = l1_table[offset / cluster_size] * 512 + (offset % cluster_size)

View file

@ -69,7 +69,7 @@ Standard options:
a description of character device properties. A common character device
definition configures a UNIX domain socket::
--chardev socket,id=char1,path=/tmp/qmp.sock,server=on,wait=off
--chardev socket,id=char1,path=/var/run/qsd-qmp.sock,server=on,wait=off
.. option:: --export [type=]nbd,id=<id>,node-name=<node-name>[,name=<export-name>][,writable=on|off][,bitmap=<name>]
--export [type=]vhost-user-blk,id=<id>,node-name=<node-name>,addr.type=unix,addr.path=<socket-path>[,writable=on|off][,logical-block-size=<block-size>][,num-queues=<num-queues>]
@ -80,8 +80,9 @@ Standard options:
requests for modifying data (the default is off).
The ``nbd`` export type requires ``--nbd-server`` (see below). ``name`` is
the NBD export name. ``bitmap`` is the name of a dirty bitmap reachable from
the block node, so the NBD client can use NBD_OPT_SET_META_CONTEXT with the
the NBD export name (if not specified, it defaults to the given
``node-name``). ``bitmap`` is the name of a dirty bitmap reachable from the
block node, so the NBD client can use NBD_OPT_SET_META_CONTEXT with the
metadata context name "qemu:dirty-bitmap:BITMAP" to inspect the bitmap.
The ``vhost-user-blk`` export type takes a vhost-user socket address on which
@ -101,14 +102,17 @@ Standard options:
.. option:: --nbd-server addr.type=inet,addr.host=<host>,addr.port=<port>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>]
--nbd-server addr.type=unix,addr.path=<path>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>]
--nbd-server addr.type=fd,addr.str=<fd>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>]
is a server for NBD exports. Both TCP and UNIX domain sockets are supported.
TLS encryption can be configured using ``--object`` tls-creds-* and authz-*
secrets (see below).
A listen socket can be provided via file descriptor passing (see Examples
below). TLS encryption can be configured using ``--object`` tls-creds-* and
authz-* secrets (see below).
To configure an NBD server on UNIX domain socket path ``/tmp/nbd.sock``::
To configure an NBD server on UNIX domain socket path
``/var/run/qsd-nbd.sock``::
--nbd-server addr.type=unix,addr.path=/tmp/nbd.sock
--nbd-server addr.type=unix,addr.path=/var/run/qsd-nbd.sock
.. option:: --object help
--object <type>,help
@ -118,6 +122,20 @@ Standard options:
List object properties with ``<type>,help``. See the :manpage:`qemu(1)`
manual page for a description of the object properties.
.. option:: --pidfile PATH
is the path to a file where the daemon writes its pid. This allows scripts to
stop the daemon by sending a signal::
$ kill -SIGTERM $(<path/to/qsd.pid)
A file lock is applied to the file so only one instance of the daemon can run
with a given pid file path. The daemon unlinks its pid file when terminating.
The pid file is written after chardevs, exports, and NBD servers have been
created but before accepting connections. The daemon has started successfully
when the pid file is written and clients may begin connecting.
Examples
--------
Launch the daemon with QMP monitor socket ``qmp.sock`` so clients can execute
@ -127,6 +145,42 @@ QMP commands::
--chardev socket,path=qmp.sock,server=on,wait=off,id=char1 \
--monitor chardev=char1
Launch the daemon from Python with a QMP monitor socket using file descriptor
passing so there is no need to busy wait for the QMP monitor to become
available::
#!/usr/bin/env python3
import subprocess
import socket
sock_path = '/var/run/qmp.sock'
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listen_sock:
listen_sock.bind(sock_path)
listen_sock.listen()
fd = listen_sock.fileno()
subprocess.Popen(
['qemu-storage-daemon',
'--chardev', f'socket,fd={fd},server=on,id=char1',
'--monitor', 'chardev=char1'],
pass_fds=[fd],
)
# listen_sock was automatically closed when leaving the 'with' statement
# body. If the daemon process terminated early then the following connect()
# will fail with "Connection refused" because no process has the listen
# socket open anymore. Launch errors can be detected this way.
qmp_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
qmp_sock.connect(sock_path)
...QMP interaction...
The same socket spawning approach also works with the ``--nbd-server
addr.type=fd,addr.str=<fd>`` and ``--export
type=vhost-user-blk,addr.type=fd,addr.str=<fd>`` options.
Export raw image file ``disk.img`` over NBD UNIX domain socket ``nbd.sock``::
$ qemu-storage-daemon \