Block layer patches:

- qcow2: Support for external data files
 - qcow2: Default to 4KB for the qcow2 cache entry size
 - Apply block driver whitelist for -drive format=help
 - Several qemu-iotests improvements
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJcgmYDAAoJEH8JsnLIjy/WiEgP/jirn5n4bFHDSzpofRxgpcEG
 2SoBpYtJQvAVgXmg8P1ZKeaX5yiiZpJS475ShuH6C3dnWuaHBBjvQfDkLLrh3v05
 4KJyZQUFx0WQZqkEiHxtgmE3iIOAsWe8VzBCZjsFITdp+fN8HlRjKVofyYP0y48G
 k9PzMlEe30Wu2s+lq6PEIXgpwIZqOVl1V3C7wDF8Vg/1OOsv+rK0vKD8yra/oQAc
 mthM+hjkpa+ohjE5aTeCsDVxO56AStvXv0d3bE0aF/ZtCvQdbVh5coYj1ldNz6VY
 Bvx+UP7kPHJw0wesZJXLeVyZUMyHAuu9vW6zKlDYJ7PoIXSLXC7rYyoEofvkAzQL
 awSluFj4HpYU1dKseJ8LzrMyUqjJ2eLJ+K48iNIh0vNlVuvX8aR62dIv0Obo3rod
 Y7zgyd6mSgDGulDa3xVsD0+eAUUEbLaUDBV80+M7S2/V9YP4Bt7lKbm0SQB4dUxm
 eOJfnbMTpIYUUjYtpCkURED3MlTIA51fy28O87TMJwa11sJTXRscysE9oNQNsvwW
 2UPhMPLykMI023glhV0vCwgXQ5kktOvpaB3U7LGQhhHd1ed9sdLEB1bO9eKWYr4N
 h6QPLBRLGUYd+BFMMIBLQtx8r+mAn1hUZoX4zxTn0lD50Rch3pRnCjMoC1iWqRkS
 J204Lzum5kgNPheyVxhy
 =GCRI
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qcow2: Support for external data files
- qcow2: Default to 4KB for the qcow2 cache entry size
- Apply block driver whitelist for -drive format=help
- Several qemu-iotests improvements

# gpg: Signature made Fri 08 Mar 2019 12:54:27 GMT
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (33 commits)
  qcow2 spec: Describe string header extensions
  qemu-iotests: Add dependency to qemu-nbd tool
  ahci-test: Add dependency to qemu-img tool
  qemu-iotests: amend with external data file
  qemu-iotests: General tests for qcow2 with external data file
  qemu-iotests: Preallocation with external data file
  qcow2: Implement data-file-raw create option
  qcow2: Store data file name in the image
  qcow2: Creating images with external data file
  qcow2: Add basic data-file infrastructure
  qcow2: Support external data file in qemu-img check
  qcow2: Return error for snapshot operation with data file
  qcow2: External file I/O
  qcow2: Prepare qcow2_co_block_status() for data file
  qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset()
  qcow2: Don't assume 0 is an invalid cluster offset
  qcow2: Prepare count_contiguous_clusters() for external data file
  qcow2: Prepare qcow2_get_cluster_type() for external data file
  qcow2: Pass bs to qcow2_get_cluster_type()
  qcow2: Basic definitions for external data files
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
This commit is contained in:
Peter Maydell 2019-03-09 14:43:39 +00:00
commit 4c76137484
215 changed files with 1583 additions and 421 deletions

View file

@ -97,7 +97,19 @@ in the description of a field.
be written to (unless for regaining
consistency).
Bits 2-63: Reserved (set to 0)
Bit 2: External data file bit. If this bit is set, an
external data file is used. Guest clusters are
then stored in the external data file. For such
images, clusters in the external data file are
not refcounted. The offset field in the
Standard Cluster Descriptor must match the
guest offset and neither compressed clusters
nor internal snapshots are supported.
An External Data File Name header extension may
be present if this bit is set.
Bits 3-63: Reserved (set to 0)
80 - 87: compatible_features
Bitmask of compatible features. An implementation can
@ -126,7 +138,21 @@ in the description of a field.
bit is unset, the bitmaps extension data must be
considered inconsistent.
Bits 1-63: Reserved (set to 0)
Bit 1: If this bit is set, the external data file can
be read as a consistent standalone raw image
without looking at the qcow2 metadata.
Setting this bit has a performance impact for
some operations on the image (e.g. writing
zeros requires writing to the data file instead
of only setting the zero flag in the L2 table
entry) and conflicts with backing files.
This bit may only be set if the External Data
File bit (incompatible feature bit 1) is also
set.
Bits 2-63: Reserved (set to 0)
96 - 99: refcount_order
Describes the width of a reference count block entry (width
@ -144,10 +170,11 @@ be stored. Each extension has a structure like the following:
Byte 0 - 3: Header extension type:
0x00000000 - End of the header extension area
0xE2792ACA - Backing file format name
0xE2792ACA - Backing file format name string
0x6803f857 - Feature name table
0x23852875 - Bitmaps extension
0x0537be77 - Full disk encryption header pointer
0x44415441 - External data file name string
other - Unknown header extension, can be safely
ignored
@ -169,6 +196,16 @@ data of compatible features that it doesn't support. Compatible features that
need space for additional data can use a header extension.
== String header extensions ==
Some header extensions (such as the backing file format name and the external
data file name) are just a single string. In this case, the header extension
length is the string length and the string is not '\0' terminated. (The header
extension padding can make it look like a string is '\0' terminated, but
neither is padding always necessary nor is there a guarantee that zero bytes
are used for padding.)
== Feature name table ==
The feature name table is an optional header extension that contains the name
@ -437,6 +474,11 @@ L2 table entry:
This information is only accurate in L2 tables
that are reachable from the active L1 table.
With external data files, all guest clusters have an
implicit refcount of 1 (because of the fixed host = guest
mapping for guest cluster offsets), so this bit should be 1
for all allocated clusters.
Standard Cluster Descriptor:
Bit 0: If set to 1, the cluster reads as all zeros. The host
@ -450,8 +492,10 @@ Standard Cluster Descriptor:
1 - 8: Reserved (set to 0)
9 - 55: Bits 9-55 of host cluster offset. Must be aligned to a
cluster boundary. If the offset is 0, the cluster is
unallocated.
cluster boundary. If the offset is 0 and bit 63 is clear,
the cluster is unallocated. The offset may only be 0 with
bit 63 set (indicating a host cluster offset of 0) when an
external data file is used.
56 - 61: Reserved (set to 0)

View file

@ -158,10 +158,10 @@ refcount cache is as small as possible unless overridden by the user.
Using smaller cache entries
---------------------------
The qcow2 L2 cache stores complete tables by default. This means that
if QEMU needs an entry from an L2 table then the whole table is read
from disk and is kept in the cache. If the cache is full then a
complete table needs to be evicted first.
The qcow2 L2 cache can store complete tables. This means that if QEMU
needs an entry from an L2 table then the whole table is read from disk
and is kept in the cache. If the cache is full then a complete table
needs to be evicted first.
This can be inefficient with large cluster sizes since it results in
more disk I/O and wastes more cache memory.
@ -172,6 +172,9 @@ it smaller than the cluster size. This can be configured using the
-drive file=hd.qcow2,l2-cache-size=2097152,l2-cache-entry-size=4096
Since QEMU 4.0 the value of l2-cache-entry-size defaults to 4KB (or
the cluster size if it's smaller).
Some things to take into account:
- The L2 cache entry size has the same restrictions as the cluster
@ -185,7 +188,8 @@ Some things to take into account:
- Try different entry sizes to see which one gives faster performance
in your case. The block size of the host filesystem is generally a
good default (usually 4096 bytes in the case of ext4).
good default (usually 4096 bytes in the case of ext4, hence the
default).
- Only the L2 cache can be configured this way. The refcount cache
always uses the cluster size as the entry size.
@ -194,7 +198,8 @@ Some things to take into account:
(as explained in the "Choosing the right cache sizes" and "How to
configure the cache sizes" sections in this document) then none of
this is necessary and you can omit the "l2-cache-entry-size"
parameter altogether.
parameter altogether. In this case QEMU makes the entry size
equal to the cluster size by default.
Reducing the memory usage