* Bug fixes and some small improvements for functional tests

* Improve performance of s390x PCI passthrough devices with relaxed translation
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmfK3dsRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbUfDQ/8CopnzCKGKFhyM5skrHbhDbUVbul6yV4L
 kIOo7N8OlrNcQB90bj+Udy+mUANHjkmSiBa5lJ/78ej4DFS6CxeVgrl1fSEl36xn
 GjWDwSUiN8pG1O4YtnDqWVTBieGSzbkQr1jHgpeAnvv08s+TtmudP1T8IznWU2v9
 FqD78SdebZ0Kua+ksBgMxwkHd6VMw13vsu6KuT9VBhie40LcDrFOuG8RDz/qo4IO
 Yg9s1Bqcy7Wa4+0ldMXS1plSdIqJBtVc/HDTg1QwH994b4Lvr7ffrFZmuVcd2dbE
 XKQ5jAMOYZqWdlXszkyd8moYGhmevCkQlALhpnbHixnfakfFYX0wTiJB6oCthFQ0
 It0J/ntNsCmJiIHNbPLzsJ1pE5+ureRnGbxVe05n+zfm8MaXL6s4nSdZzHyp8n43
 UZQqVzK55Q34K9O0qoUdCdBCjMKS9v5u95jjJo8+nc8sJoeQTssOoiixwB/E4y21
 9qSh7CbDjQK4zwuzQ7jKD603zAJH6ivvsHXlMBMXJFBiSMCAoQQ1vyou6yRHswRr
 gLHDwiWUx8SX8ckbbJ/+Zo9+T8JBMvC5hNYG8VoAtlTQusG4bHSbKdPNNH0eHsEp
 f7RlZPRizkcK3w0Nj+u4kXdnnex3QLLSgnyAYq7zEl6V+mho8KqaBezkO7wQDHZy
 +GW5ignQ1Gs=
 =CyiZ
 -----END PGP SIGNATURE-----

Merge tag 'pull-request-2025-03-07' of https://gitlab.com/thuth/qemu into staging

* Bug fixes and some small improvements for functional tests
* Improve performance of s390x PCI passthrough devices with relaxed translation

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmfK3dsRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbUfDQ/8CopnzCKGKFhyM5skrHbhDbUVbul6yV4L
# kIOo7N8OlrNcQB90bj+Udy+mUANHjkmSiBa5lJ/78ej4DFS6CxeVgrl1fSEl36xn
# GjWDwSUiN8pG1O4YtnDqWVTBieGSzbkQr1jHgpeAnvv08s+TtmudP1T8IznWU2v9
# FqD78SdebZ0Kua+ksBgMxwkHd6VMw13vsu6KuT9VBhie40LcDrFOuG8RDz/qo4IO
# Yg9s1Bqcy7Wa4+0ldMXS1plSdIqJBtVc/HDTg1QwH994b4Lvr7ffrFZmuVcd2dbE
# XKQ5jAMOYZqWdlXszkyd8moYGhmevCkQlALhpnbHixnfakfFYX0wTiJB6oCthFQ0
# It0J/ntNsCmJiIHNbPLzsJ1pE5+ureRnGbxVe05n+zfm8MaXL6s4nSdZzHyp8n43
# UZQqVzK55Q34K9O0qoUdCdBCjMKS9v5u95jjJo8+nc8sJoeQTssOoiixwB/E4y21
# 9qSh7CbDjQK4zwuzQ7jKD603zAJH6ivvsHXlMBMXJFBiSMCAoQQ1vyou6yRHswRr
# gLHDwiWUx8SX8ckbbJ/+Zo9+T8JBMvC5hNYG8VoAtlTQusG4bHSbKdPNNH0eHsEp
# f7RlZPRizkcK3w0Nj+u4kXdnnex3QLLSgnyAYq7zEl6V+mho8KqaBezkO7wQDHZy
# +GW5ignQ1Gs=
# =CyiZ
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 07 Mar 2025 19:51:55 HKT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2025-03-07' of https://gitlab.com/thuth/qemu:
  s390x/pci: indicate QEMU supports relaxed translation for passthrough
  s390x/pci: add support for guests that request direct mapping
  MAINTAINERS: Add docs/devel/testing/functional.rst to the functional section
  doc: add missing 'Asset' type in function test doc
  tests/functional/test_virtio_balloon: Only use KVM for running this test
  tests/functional: fix race in virtio balloon test
  tests/functional: Increase the timeout of the mips64el_replay test
  tests/functional/test_mips_malta: Add a network test via the pcnet NIC
  tests/functional: Move the code for testing HTTP downloads to a common function
  tests/functional: stop output from zstd command when uncompressing
  tests/functional: drop unused 'get_tag' method
  tests/functional: skip memaddr tests on 32-bit builds
  tests/functional: reduce tuxrun maxmem to work on 32-bit hosts
  tests/functional: set 'qemu_bin' as an object level field
  tests/functional: remove unused 'bin_prefix' variable

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
This commit is contained in:
Stefan Hajnoczi 2025-03-08 11:31:26 +08:00
commit e88a579392
18 changed files with 198 additions and 61 deletions

View file

@ -4236,6 +4236,7 @@ Functional testing framework
M: Thomas Huth <thuth@redhat.com> M: Thomas Huth <thuth@redhat.com>
R: Philippe Mathieu-Daudé <philmd@linaro.org> R: Philippe Mathieu-Daudé <philmd@linaro.org>
R: Daniel P. Berrange <berrange@redhat.com> R: Daniel P. Berrange <berrange@redhat.com>
F: docs/devel/testing/functional.rst
F: tests/functional/qemu_test/ F: tests/functional/qemu_test/
Windows Hosted Continuous Integration Windows Hosted Continuous Integration

View file

@ -173,7 +173,7 @@ QEMU binary selection
^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^
The QEMU binary used for the ``self.vm`` QEMUMachine instance will The QEMU binary used for the ``self.vm`` QEMUMachine instance will
primarily depend on the value of the ``qemu_bin`` class attribute. primarily depend on the value of the ``qemu_bin`` instance attribute.
If it is not explicitly set by the test code, its default value will If it is not explicitly set by the test code, its default value will
be the result the QEMU_TEST_QEMU_BINARY environment variable. be the result the QEMU_TEST_QEMU_BINARY environment variable.
@ -251,7 +251,7 @@ Many functional tests download assets (e.g. Linux kernels, initrds,
firmware images, etc.) from the internet to be able to run tests with firmware images, etc.) from the internet to be able to run tests with
them. This imposes additional challenges to the test framework. them. This imposes additional challenges to the test framework.
First there is the the problem that some people might not have an First there is the problem that some people might not have an
unconstrained internet connection, so such tests should not be run by unconstrained internet connection, so such tests should not be run by
default when running ``make check``. To accomplish this situation, default when running ``make check``. To accomplish this situation,
the tests that download files should only be added to the "thorough" the tests that download files should only be added to the "thorough"
@ -274,7 +274,9 @@ the tests are run. This pre-caching is done with the qemu_test.Asset
class. To use it in your test, declare an asset in your test class with class. To use it in your test, declare an asset in your test class with
its URL and SHA256 checksum like this:: its URL and SHA256 checksum like this::
ASSET_somename = ( from qemu_test import Asset
ASSET_somename = Asset(
('https://www.qemu.org/assets/images/qemu_head_200.png'), ('https://www.qemu.org/assets/images/qemu_head_200.png'),
'34b74cad46ea28a2966c1d04e102510daf1fd73e6582b6b74523940d5da029dd') '34b74cad46ea28a2966c1d04e102510daf1fd73e6582b6b74523940d5da029dd')

View file

@ -18,6 +18,8 @@
#include "hw/s390x/s390-pci-inst.h" #include "hw/s390x/s390-pci-inst.h"
#include "hw/s390x/s390-pci-kvm.h" #include "hw/s390x/s390-pci-kvm.h"
#include "hw/s390x/s390-pci-vfio.h" #include "hw/s390x/s390-pci-vfio.h"
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/boards.h"
#include "hw/pci/pci_bus.h" #include "hw/pci/pci_bus.h"
#include "hw/qdev-properties.h" #include "hw/qdev-properties.h"
#include "hw/pci/pci_bridge.h" #include "hw/pci/pci_bridge.h"
@ -724,12 +726,42 @@ void s390_pci_iommu_enable(S390PCIIOMMU *iommu)
g_free(name); g_free(name);
} }
void s390_pci_iommu_direct_map_enable(S390PCIIOMMU *iommu)
{
MachineState *ms = MACHINE(qdev_get_machine());
S390CcwMachineState *s390ms = S390_CCW_MACHINE(ms);
/*
* For direct-mapping we must map the entire guest address space. Rather
* than using an iommu, create a memory region alias that maps GPA X to
* IOVA X + SDMA. VFIO will handle pinning via its memory listener.
*/
g_autofree char *name = g_strdup_printf("iommu-dm-s390-%04x",
iommu->pbdev->uid);
iommu->dm_mr = g_malloc0(sizeof(*iommu->dm_mr));
memory_region_init_alias(iommu->dm_mr, OBJECT(&iommu->mr), name,
get_system_memory(), 0,
s390_get_memory_limit(s390ms));
iommu->enabled = true;
memory_region_add_subregion(&iommu->mr, iommu->pbdev->zpci_fn.sdma,
iommu->dm_mr);
}
void s390_pci_iommu_disable(S390PCIIOMMU *iommu) void s390_pci_iommu_disable(S390PCIIOMMU *iommu)
{ {
iommu->enabled = false; iommu->enabled = false;
g_hash_table_remove_all(iommu->iotlb); g_hash_table_remove_all(iommu->iotlb);
memory_region_del_subregion(&iommu->mr, MEMORY_REGION(&iommu->iommu_mr)); if (iommu->dm_mr) {
memory_region_del_subregion(&iommu->mr, iommu->dm_mr);
object_unparent(OBJECT(iommu->dm_mr));
g_free(iommu->dm_mr);
iommu->dm_mr = NULL;
} else {
memory_region_del_subregion(&iommu->mr,
MEMORY_REGION(&iommu->iommu_mr));
object_unparent(OBJECT(&iommu->iommu_mr)); object_unparent(OBJECT(&iommu->iommu_mr));
}
} }
static void s390_pci_iommu_free(S390pciState *s, PCIBus *bus, int32_t devfn) static void s390_pci_iommu_free(S390pciState *s, PCIBus *bus, int32_t devfn)
@ -1145,6 +1177,7 @@ static void s390_pcihost_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
/* Always intercept emulated devices */ /* Always intercept emulated devices */
pbdev->interp = false; pbdev->interp = false;
pbdev->forwarding_assist = false; pbdev->forwarding_assist = false;
pbdev->rtr_avail = false;
} }
if (s390_pci_msix_init(pbdev) && !pbdev->interp) { if (s390_pci_msix_init(pbdev) && !pbdev->interp) {
@ -1511,6 +1544,8 @@ static const Property s390_pci_device_properties[] = {
DEFINE_PROP_BOOL("interpret", S390PCIBusDevice, interp, true), DEFINE_PROP_BOOL("interpret", S390PCIBusDevice, interp, true),
DEFINE_PROP_BOOL("forwarding-assist", S390PCIBusDevice, forwarding_assist, DEFINE_PROP_BOOL("forwarding-assist", S390PCIBusDevice, forwarding_assist,
true), true),
DEFINE_PROP_BOOL("relaxed-translation", S390PCIBusDevice, rtr_avail,
true),
}; };
static const VMStateDescription s390_pci_device_vmstate = { static const VMStateDescription s390_pci_device_vmstate = {

View file

@ -16,6 +16,7 @@
#include "exec/memory.h" #include "exec/memory.h"
#include "qemu/error-report.h" #include "qemu/error-report.h"
#include "system/hw_accel.h" #include "system/hw_accel.h"
#include "hw/boards.h"
#include "hw/pci/pci_device.h" #include "hw/pci/pci_device.h"
#include "hw/s390x/s390-pci-inst.h" #include "hw/s390x/s390-pci-inst.h"
#include "hw/s390x/s390-pci-bus.h" #include "hw/s390x/s390-pci-bus.h"
@ -1008,17 +1009,25 @@ static int reg_ioat(CPUS390XState *env, S390PCIBusDevice *pbdev, ZpciFib fib,
} }
/* currently we only support designation type 1 with translation */ /* currently we only support designation type 1 with translation */
if (!(dt == ZPCI_IOTA_RTTO && t)) { if (t && dt != ZPCI_IOTA_RTTO) {
error_report("unsupported ioat dt %d t %d", dt, t); error_report("unsupported ioat dt %d t %d", dt, t);
s390_program_interrupt(env, PGM_OPERAND, ra); s390_program_interrupt(env, PGM_OPERAND, ra);
return -EINVAL; return -EINVAL;
} else if (!t && !pbdev->rtr_avail) {
error_report("relaxed translation not allowed");
s390_program_interrupt(env, PGM_OPERAND, ra);
return -EINVAL;
} }
iommu->pba = pba; iommu->pba = pba;
iommu->pal = pal; iommu->pal = pal;
iommu->g_iota = g_iota; iommu->g_iota = g_iota;
if (t) {
s390_pci_iommu_enable(iommu); s390_pci_iommu_enable(iommu);
} else {
s390_pci_iommu_direct_map_enable(iommu);
}
return 0; return 0;
} }

View file

@ -132,13 +132,28 @@ static void s390_pci_read_base(S390PCIBusDevice *pbdev,
pbdev->pft = cap->pft; pbdev->pft = cap->pft;
/* /*
* If appropriate, reduce the size of the supported DMA aperture reported * If the device is a passthrough ISM device, disallow relaxed
* to the guest based upon the vfio DMA limit. * translation.
*/ */
if (pbdev->pft == ZPCI_PFT_ISM) {
pbdev->rtr_avail = false;
}
/*
* If appropriate, reduce the size of the supported DMA aperture reported
* to the guest based upon the vfio DMA limit. This is applicable for
* devices that are guaranteed to not use relaxed translation. If the
* device is capable of relaxed translation then we must advertise the
* full aperture. In this case, if translation is used then we will
* rely on the vfio DMA limit counting and use RPCIT CC1 / status 16
* to request that the guest free DMA mappings as necessary.
*/
if (!pbdev->rtr_avail) {
vfio_size = pbdev->iommu->max_dma_limit << TARGET_PAGE_BITS; vfio_size = pbdev->iommu->max_dma_limit << TARGET_PAGE_BITS;
if (vfio_size > 0 && vfio_size < cap->end_dma - cap->start_dma + 1) { if (vfio_size > 0 && vfio_size < cap->end_dma - cap->start_dma + 1) {
pbdev->zpci_fn.edma = cap->start_dma + vfio_size - 1; pbdev->zpci_fn.edma = cap->start_dma + vfio_size - 1;
} }
}
} }
static bool get_host_fh(S390PCIBusDevice *pbdev, struct vfio_device_info *info, static bool get_host_fh(S390PCIBusDevice *pbdev, struct vfio_device_info *info,
@ -223,8 +238,11 @@ static void s390_pci_read_group(S390PCIBusDevice *pbdev,
pbdev->pci_group = s390_group_create(pbdev->zpci_fn.pfgid, start_gid); pbdev->pci_group = s390_group_create(pbdev->zpci_fn.pfgid, start_gid);
resgrp = &pbdev->pci_group->zpci_group; resgrp = &pbdev->pci_group->zpci_group;
if (pbdev->rtr_avail) {
resgrp->fr |= CLP_RSP_QPCIG_MASK_RTR;
}
if (cap->flags & VFIO_DEVICE_INFO_ZPCI_FLAG_REFRESH) { if (cap->flags & VFIO_DEVICE_INFO_ZPCI_FLAG_REFRESH) {
resgrp->fr = 1; resgrp->fr |= CLP_RSP_QPCIG_MASK_REFRESH;
} }
resgrp->dasm = cap->dasm; resgrp->dasm = cap->dasm;
resgrp->msia = cap->msi_addr; resgrp->msia = cap->msi_addr;

View file

@ -936,8 +936,13 @@ static void ccw_machine_9_2_instance_options(MachineState *machine)
static void ccw_machine_9_2_class_options(MachineClass *mc) static void ccw_machine_9_2_class_options(MachineClass *mc)
{ {
static GlobalProperty compat[] = {
{ TYPE_S390_PCI_DEVICE, "relaxed-translation", "off", },
};
ccw_machine_10_0_class_options(mc); ccw_machine_10_0_class_options(mc);
compat_props_add(mc->compat_props, hw_compat_9_2, hw_compat_9_2_len); compat_props_add(mc->compat_props, hw_compat_9_2, hw_compat_9_2_len);
compat_props_add(mc->compat_props, compat, G_N_ELEMENTS(compat));
} }
DEFINE_CCW_MACHINE(9, 2); DEFINE_CCW_MACHINE(9, 2);

View file

@ -277,6 +277,7 @@ struct S390PCIIOMMU {
AddressSpace as; AddressSpace as;
MemoryRegion mr; MemoryRegion mr;
IOMMUMemoryRegion iommu_mr; IOMMUMemoryRegion iommu_mr;
MemoryRegion *dm_mr;
bool enabled; bool enabled;
uint64_t g_iota; uint64_t g_iota;
uint64_t pba; uint64_t pba;
@ -362,6 +363,7 @@ struct S390PCIBusDevice {
bool interp; bool interp;
bool forwarding_assist; bool forwarding_assist;
bool aif; bool aif;
bool rtr_avail;
QTAILQ_ENTRY(S390PCIBusDevice) link; QTAILQ_ENTRY(S390PCIBusDevice) link;
}; };
@ -389,6 +391,7 @@ int pci_chsc_sei_nt2_have_event(void);
void s390_pci_sclp_configure(SCCB *sccb); void s390_pci_sclp_configure(SCCB *sccb);
void s390_pci_sclp_deconfigure(SCCB *sccb); void s390_pci_sclp_deconfigure(SCCB *sccb);
void s390_pci_iommu_enable(S390PCIIOMMU *iommu); void s390_pci_iommu_enable(S390PCIIOMMU *iommu);
void s390_pci_iommu_direct_map_enable(S390PCIIOMMU *iommu);
void s390_pci_iommu_disable(S390PCIIOMMU *iommu); void s390_pci_iommu_disable(S390PCIIOMMU *iommu);
void s390_pci_generate_error_event(uint16_t pec, uint32_t fh, uint32_t fid, void s390_pci_generate_error_event(uint16_t pec, uint32_t fh, uint32_t fid,
uint64_t faddr, uint32_t e); uint64_t faddr, uint32_t e);

View file

@ -158,6 +158,7 @@ typedef struct ClpRspQueryPciGrp {
#define CLP_RSP_QPCIG_MASK_NOI 0xfff #define CLP_RSP_QPCIG_MASK_NOI 0xfff
uint16_t i; uint16_t i;
uint8_t version; uint8_t version;
#define CLP_RSP_QPCIG_MASK_RTR 0x20
#define CLP_RSP_QPCIG_MASK_FRAME 0x2 #define CLP_RSP_QPCIG_MASK_FRAME 0x2
#define CLP_RSP_QPCIG_MASK_REFRESH 0x1 #define CLP_RSP_QPCIG_MASK_REFRESH 0x1
uint8_t fr; uint8_t fr;

View file

@ -36,6 +36,7 @@ test_timeouts = {
'intel_iommu': 300, 'intel_iommu': 300,
'mips_malta' : 120, 'mips_malta' : 120,
'mipsel_replay' : 480, 'mipsel_replay' : 480,
'mips64el_replay' : 180,
'netdev_ethtool' : 180, 'netdev_ethtool' : 180,
'ppc_40p' : 240, 'ppc_40p' : 240,
'ppc64_hv' : 1000, 'ppc64_hv' : 1000,

View file

@ -3,8 +3,12 @@
# This work is licensed under the terms of the GNU GPL, version 2 or # This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory. # later. See the COPYING file in the top-level directory.
import hashlib
import urllib.request
from .cmd import wait_for_console_pattern, exec_command_and_wait_for_pattern
from .testcase import QemuSystemTest from .testcase import QemuSystemTest
from .cmd import wait_for_console_pattern from .utils import get_usernet_hostfwd_port
class LinuxKernelTest(QemuSystemTest): class LinuxKernelTest(QemuSystemTest):
@ -26,3 +30,23 @@ class LinuxKernelTest(QemuSystemTest):
self.vm.launch() self.vm.launch()
if wait_for: if wait_for:
self.wait_for_console_pattern(wait_for) self.wait_for_console_pattern(wait_for)
def check_http_download(self, filename, hashsum, guestport=8080,
pythoncmd='python3 -m http.server'):
exec_command_and_wait_for_pattern(self,
f'{pythoncmd} {guestport} & sleep 1',
f'Serving HTTP on 0.0.0.0 port {guestport}')
hl = hashlib.sha256()
hostport = get_usernet_hostfwd_port(self.vm)
url = f'http://localhost:{hostport}{filename}'
self.log.info(f'Downloading {url} ...')
with urllib.request.urlopen(url) as response:
while True:
chunk = response.read(1 << 20)
if not chunk:
break
hl.update(chunk)
digest = hl.hexdigest()
self.log.info(f'sha256sum of download is {digest}.')
self.assertEqual(digest, hashsum)

View file

@ -33,7 +33,6 @@ from .uncompress import uncompress
class QemuBaseTest(unittest.TestCase): class QemuBaseTest(unittest.TestCase):
qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
arch = None arch = None
workdir = None workdir = None
@ -192,7 +191,8 @@ class QemuBaseTest(unittest.TestCase):
return False return False
return True return True
def setUp(self, bin_prefix): def setUp(self):
self.qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
self.arch = self.qemu_bin.split('-')[-1] self.arch = self.qemu_bin.split('-')[-1]
self.socketdir = None self.socketdir = None
@ -254,7 +254,7 @@ class QemuBaseTest(unittest.TestCase):
class QemuUserTest(QemuBaseTest): class QemuUserTest(QemuBaseTest):
def setUp(self): def setUp(self):
super().setUp('qemu-') super().setUp()
self._ldpath = [] self._ldpath = []
def add_ldpath(self, ldpath): def add_ldpath(self, ldpath):
@ -277,7 +277,7 @@ class QemuSystemTest(QemuBaseTest):
def setUp(self): def setUp(self):
self._vms = {} self._vms = {}
super().setUp('qemu-system-') super().setUp()
console_log = logging.getLogger('console') console_log = logging.getLogger('console')
console_log.setLevel(logging.DEBUG) console_log.setLevel(logging.DEBUG)

View file

@ -24,17 +24,6 @@ class TuxRunBaselineTest(QemuSystemTest):
# Tests are ~10-40s, allow for --debug/--enable-gcov overhead # Tests are ~10-40s, allow for --debug/--enable-gcov overhead
timeout = 100 timeout = 100
def get_tag(self, tagname, default=None):
"""
Get the metadata tag or return the default.
"""
utag = self._get_unique_tag_val(tagname)
print(f"{tagname}/{default} -> {utag}")
if utag:
return utag
return default
def setUp(self): def setUp(self):
super().setUp() super().setUp()

View file

@ -13,7 +13,7 @@ import os
import stat import stat
import shutil import shutil
from urllib.parse import urlparse from urllib.parse import urlparse
from subprocess import check_call, CalledProcessError from subprocess import run, CalledProcessError, DEVNULL
from .asset import Asset from .asset import Asset
@ -46,8 +46,8 @@ def zstd_uncompress(zstd_path, output_path):
return return
try: try:
check_call(['zstd', "-f", "-d", zstd_path, run(['zstd', "-f", "-d", zstd_path,
"-o", output_path]) "-o", output_path], capture_output=True, check=True)
except CalledProcessError as e: except CalledProcessError as e:
os.remove(output_path) os.remove(output_path)
raise Exception( raise Exception(

View file

@ -10,11 +10,7 @@
# This work is licensed under the terms of the GNU GPL, version 2 or # This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory. # later. See the COPYING file in the top-level directory.
import hashlib
import urllib.request
from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern
from qemu_test.utils import get_usernet_hostfwd_port
class IntelIOMMU(LinuxKernelTest): class IntelIOMMU(LinuxKernelTest):
@ -125,23 +121,7 @@ class IntelIOMMU(LinuxKernelTest):
# Check virtio-net via HTTP: # Check virtio-net via HTTP:
exec_command_and_wait_for_pattern(self, 'dhclient eth0', prompt) exec_command_and_wait_for_pattern(self, 'dhclient eth0', prompt)
exec_command_and_wait_for_pattern(self, self.check_http_download(filename, hashsum, self.GUEST_PORT)
f'python3 -m http.server {self.GUEST_PORT} & sleep 1',
f'Serving HTTP on 0.0.0.0 port {self.GUEST_PORT}')
hl = hashlib.sha256()
hostport = get_usernet_hostfwd_port(self.vm)
url = f'http://localhost:{hostport}{filename}'
self.log.info(f'Downloading {url} ...')
with urllib.request.urlopen(url) as response:
while True:
chunk = response.read(1 << 20)
if not chunk:
break
hl.update(chunk)
digest = hl.hexdigest()
self.log.info(f'sha256sum of download is {digest}.')
self.assertEqual(digest, hashsum)
def test_intel_iommu(self): def test_intel_iommu(self):
self.common_vm_setup() self.common_vm_setup()

View file

@ -20,6 +20,25 @@ class MemAddrCheck(QemuSystemTest):
# this reason. # this reason.
DELAY_Q35_BOOT_SEQUENCE = 1 DELAY_Q35_BOOT_SEQUENCE = 1
# This helper can go away when the 32-bit host deprecation
# turns into full & final removal of support.
def ensure_64bit_binary(self):
with open(self.qemu_bin, "rb") as fh:
ident = fh.read(4)
# "\x7fELF"
if ident != bytes([0x7f, 0x45, 0x4C, 0x46]):
# Non-ELF file implies macOS or Windows which
# we already assume to be 64-bit only
return
# bits == 1 -> 32-bit; bits == 2 -> 64-bit
bits = int.from_bytes(fh.read(1), byteorder='little')
if bits != 2:
# 32-bit ELF builds won't be able to address sufficient
# RAM to run the tests
self.skipTest("64-bit build host is required")
# first, lets test some 32-bit processors. # first, lets test some 32-bit processors.
# for all 32-bit cases, pci64_hole_size is 0. # for all 32-bit cases, pci64_hole_size is 0.
def test_phybits_low_pse36(self): def test_phybits_low_pse36(self):
@ -38,6 +57,7 @@ class MemAddrCheck(QemuSystemTest):
If maxmem is set to 59.5G with all other QEMU parameters identical, QEMU If maxmem is set to 59.5G with all other QEMU parameters identical, QEMU
should start fine. should start fine.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-machine', 'q35', '-m', self.vm.add_args('-S', '-machine', 'q35', '-m',
'512,slots=1,maxmem=59.6G', '512,slots=1,maxmem=59.6G',
'-cpu', 'pentium,pse36=on', '-display', 'none', '-cpu', 'pentium,pse36=on', '-display', 'none',
@ -55,6 +75,7 @@ class MemAddrCheck(QemuSystemTest):
access up to a maximum of 64GiB of memory. Rest is the same as the case access up to a maximum of 64GiB of memory. Rest is the same as the case
with pse36 above. with pse36 above.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-machine', 'q35', '-m', self.vm.add_args('-S', '-machine', 'q35', '-m',
'512,slots=1,maxmem=59.6G', '512,slots=1,maxmem=59.6G',
'-cpu', 'pentium,pae=on', '-display', 'none', '-cpu', 'pentium,pae=on', '-display', 'none',
@ -71,6 +92,7 @@ class MemAddrCheck(QemuSystemTest):
Setting maxmem to 59.5G and making sure that QEMU can start with the Setting maxmem to 59.5G and making sure that QEMU can start with the
same options as the failing case above with pse36 cpu feature. same options as the failing case above with pse36 cpu feature.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-machine', 'q35', '-m', self.vm.add_args('-machine', 'q35', '-m',
'512,slots=1,maxmem=59.5G', '512,slots=1,maxmem=59.5G',
'-cpu', 'pentium,pse36=on', '-display', 'none', '-cpu', 'pentium,pse36=on', '-display', 'none',
@ -88,6 +110,7 @@ class MemAddrCheck(QemuSystemTest):
Setting maxmem to 59.5G and making sure that QEMU can start fine Setting maxmem to 59.5G and making sure that QEMU can start fine
with the same options as the case above. with the same options as the case above.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-machine', 'q35', '-m', self.vm.add_args('-machine', 'q35', '-m',
'512,slots=1,maxmem=59.5G', '512,slots=1,maxmem=59.5G',
'-cpu', 'pentium,pae=on', '-display', 'none', '-cpu', 'pentium,pae=on', '-display', 'none',
@ -104,6 +127,7 @@ class MemAddrCheck(QemuSystemTest):
Pentium2 has 36 bits of addressing, so its same as pentium Pentium2 has 36 bits of addressing, so its same as pentium
with pse36 ON. with pse36 ON.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-machine', 'q35', '-m', self.vm.add_args('-machine', 'q35', '-m',
'512,slots=1,maxmem=59.5G', '512,slots=1,maxmem=59.5G',
'-cpu', 'pentium2', '-display', 'none', '-cpu', 'pentium2', '-display', 'none',
@ -123,6 +147,7 @@ class MemAddrCheck(QemuSystemTest):
message because the region for memory hotplug is always placed message because the region for memory hotplug is always placed
above 4 GiB due to the PCI hole and simplicity. above 4 GiB due to the PCI hole and simplicity.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-machine', 'q35', '-m', self.vm.add_args('-S', '-machine', 'q35', '-m',
'512,slots=1,maxmem=4G', '512,slots=1,maxmem=4G',
'-cpu', 'pentium', '-display', 'none', '-cpu', 'pentium', '-display', 'none',
@ -150,6 +175,7 @@ class MemAddrCheck(QemuSystemTest):
which is equal to 987.5 GiB. Setting the value to 988 GiB should which is equal to 987.5 GiB. Setting the value to 988 GiB should
make QEMU fail with the error message. make QEMU fail with the error message.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-machine', 'pc-q35-7.0', '-m', self.vm.add_args('-S', '-machine', 'pc-q35-7.0', '-m',
'512,slots=1,maxmem=988G', '512,slots=1,maxmem=988G',
'-display', 'none', '-display', 'none',
@ -170,6 +196,7 @@ class MemAddrCheck(QemuSystemTest):
Make sure QEMU fails when maxmem size is 976 GiB (12 GiB less Make sure QEMU fails when maxmem size is 976 GiB (12 GiB less
than 988 GiB). than 988 GiB).
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-machine', 'pc-q35-7.1', '-m', self.vm.add_args('-S', '-machine', 'pc-q35-7.1', '-m',
'512,slots=1,maxmem=976G', '512,slots=1,maxmem=976G',
'-display', 'none', '-display', 'none',
@ -186,6 +213,7 @@ class MemAddrCheck(QemuSystemTest):
Same as q35-7.0 AMD case except that here we check that QEMU can Same as q35-7.0 AMD case except that here we check that QEMU can
successfully start when maxmem is < 988G. successfully start when maxmem is < 988G.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-machine', 'pc-q35-7.0', '-m', self.vm.add_args('-S', '-machine', 'pc-q35-7.0', '-m',
'512,slots=1,maxmem=987.5G', '512,slots=1,maxmem=987.5G',
'-display', 'none', '-display', 'none',
@ -202,6 +230,7 @@ class MemAddrCheck(QemuSystemTest):
Same as q35-7.1 AMD case except that here we check that QEMU can Same as q35-7.1 AMD case except that here we check that QEMU can
successfully start when maxmem is < 976G. successfully start when maxmem is < 976G.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-machine', 'pc-q35-7.1', '-m', self.vm.add_args('-S', '-machine', 'pc-q35-7.1', '-m',
'512,slots=1,maxmem=975.5G', '512,slots=1,maxmem=975.5G',
'-display', 'none', '-display', 'none',
@ -219,6 +248,7 @@ class MemAddrCheck(QemuSystemTest):
Intel cpu instead. QEMU should start fine in this case as Intel cpu instead. QEMU should start fine in this case as
"above_4G" memory starts at 4G. "above_4G" memory starts at 4G.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-cpu', 'Skylake-Server', self.vm.add_args('-S', '-cpu', 'Skylake-Server',
'-machine', 'pc-q35-7.1', '-m', '-machine', 'pc-q35-7.1', '-m',
'512,slots=1,maxmem=976G', '512,slots=1,maxmem=976G',
@ -243,6 +273,7 @@ class MemAddrCheck(QemuSystemTest):
memory for the VM (1024 - 32 - 1 + 0.5). With 992 GiB, QEMU should memory for the VM (1024 - 32 - 1 + 0.5). With 992 GiB, QEMU should
fail to start. fail to start.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-cpu', 'EPYC-v4,phys-bits=41', self.vm.add_args('-S', '-cpu', 'EPYC-v4,phys-bits=41',
'-machine', 'pc-q35-7.1', '-m', '-machine', 'pc-q35-7.1', '-m',
'512,slots=1,maxmem=992G', '512,slots=1,maxmem=992G',
@ -261,6 +292,7 @@ class MemAddrCheck(QemuSystemTest):
Same as above but by setting maxram between 976 GiB and 992 Gib, Same as above but by setting maxram between 976 GiB and 992 Gib,
QEMU should start fine. QEMU should start fine.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-cpu', 'EPYC-v4,phys-bits=41', self.vm.add_args('-S', '-cpu', 'EPYC-v4,phys-bits=41',
'-machine', 'pc-q35-7.1', '-m', '-machine', 'pc-q35-7.1', '-m',
'512,slots=1,maxmem=990G', '512,slots=1,maxmem=990G',
@ -281,6 +313,7 @@ class MemAddrCheck(QemuSystemTest):
So maxmem here should be at most 986 GiB considering all memory boundary So maxmem here should be at most 986 GiB considering all memory boundary
alignment constraints with 40 bits (1 TiB) of processor physical bits. alignment constraints with 40 bits (1 TiB) of processor physical bits.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-cpu', 'Skylake-Server,phys-bits=40', self.vm.add_args('-S', '-cpu', 'Skylake-Server,phys-bits=40',
'-machine', 'q35,cxl=on', '-m', '-machine', 'q35,cxl=on', '-m',
'512,slots=1,maxmem=987G', '512,slots=1,maxmem=987G',
@ -299,6 +332,7 @@ class MemAddrCheck(QemuSystemTest):
with the exact same parameters as above, QEMU should start fine even with the exact same parameters as above, QEMU should start fine even
with cxl enabled. with cxl enabled.
""" """
self.ensure_64bit_binary()
self.vm.add_args('-S', '-cpu', 'Skylake-Server,phys-bits=40', self.vm.add_args('-S', '-cpu', 'Skylake-Server,phys-bits=40',
'-machine', 'q35,cxl=on', '-m', '-machine', 'q35,cxl=on', '-m',
'512,slots=1,maxmem=987G', '512,slots=1,maxmem=987G',

View file

@ -45,12 +45,15 @@ class MaltaMachineConsole(LinuxKernelTest):
'dcfe3a7fe3200da3a00d176b95caaa086495eb158f2bff64afc67d7e1eb2cddc') 'dcfe3a7fe3200da3a00d176b95caaa086495eb158f2bff64afc67d7e1eb2cddc')
def test_mips_malta_cpio(self): def test_mips_malta_cpio(self):
self.require_netdev('user')
self.set_machine('malta')
self.require_device('pcnet')
kernel_path = self.archive_extract( kernel_path = self.archive_extract(
self.ASSET_KERNEL_4_5_0, self.ASSET_KERNEL_4_5_0,
member='boot/vmlinux-4.5.0-2-4kc-malta') member='boot/vmlinux-4.5.0-2-4kc-malta')
initrd_path = self.uncompress(self.ASSET_INITRD) initrd_path = self.uncompress(self.ASSET_INITRD)
self.set_machine('malta')
self.vm.set_console() self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ 'console=ttyS0 console=tty ' + 'console=ttyS0 console=tty '
@ -58,6 +61,8 @@ class MaltaMachineConsole(LinuxKernelTest):
self.vm.add_args('-kernel', kernel_path, self.vm.add_args('-kernel', kernel_path,
'-initrd', initrd_path, '-initrd', initrd_path,
'-append', kernel_command_line, '-append', kernel_command_line,
'-netdev', 'user,id=n1,tftp=' + self.scratch_file('boot'),
'-device', 'pcnet,netdev=n1',
'-no-reboot') '-no-reboot')
self.vm.launch() self.vm.launch()
self.wait_for_console_pattern('Boot successful.') self.wait_for_console_pattern('Boot successful.')
@ -66,6 +71,19 @@ class MaltaMachineConsole(LinuxKernelTest):
'BogoMIPS') 'BogoMIPS')
exec_command_and_wait_for_pattern(self, 'uname -a', exec_command_and_wait_for_pattern(self, 'uname -a',
'Debian') 'Debian')
exec_command_and_wait_for_pattern(self, 'ip link set eth0 up',
'eth0: link up')
exec_command_and_wait_for_pattern(self,
'ip addr add 10.0.2.15 dev eth0',
'#')
exec_command_and_wait_for_pattern(self, 'route add default eth0', '#')
exec_command_and_wait_for_pattern(self,
'tftp -g -r vmlinux-4.5.0-2-4kc-malta 10.0.2.2', '#')
exec_command_and_wait_for_pattern(self,
'md5sum vmlinux-4.5.0-2-4kc-malta',
'a98218a7efbdefb2dfdf9ecd08c98318')
exec_command_and_wait_for_pattern(self, 'reboot', exec_command_and_wait_for_pattern(self, 'reboot',
'reboot: Restarting system') 'reboot: Restarting system')
# Wait for VM to shut down gracefully # Wait for VM to shut down gracefully

View file

@ -64,7 +64,7 @@ class TuxRunPPC64Test(TuxRunBaselineTest):
',"index":1,"id":"pci.1"}') ',"index":1,"id":"pci.1"}')
self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
',"reg":12288}') ',"reg":12288}')
self.vm.add_args('-m', '2G,slots=32,maxmem=4G', self.vm.add_args('-m', '1G,slots=32,maxmem=2G',
'-object', 'memory-backend-ram,id=ram1,size=1G', '-object', 'memory-backend-ram,id=ram1,size=1G',
'-device', 'pc-dimm,id=dimm1,memdev=ram1') '-device', 'pc-dimm,id=dimm1,memdev=ram1')

View file

@ -32,7 +32,7 @@ class VirtioBalloonx86(QemuSystemTest):
'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0') 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0')
DEFAULT_KERNEL_PARAMS = ('root=/dev/vda1 console=ttyS0 net.ifnames=0 ' DEFAULT_KERNEL_PARAMS = ('root=/dev/vda1 console=ttyS0 net.ifnames=0 '
'rd.rescue') 'rd.rescue quiet')
def wait_for_console_pattern(self, success_message, vm=None): def wait_for_console_pattern(self, success_message, vm=None):
wait_for_console_pattern( wait_for_console_pattern(
@ -47,6 +47,11 @@ class VirtioBalloonx86(QemuSystemTest):
prompt = '# ' prompt = '# '
self.wait_for_console_pattern(prompt) self.wait_for_console_pattern(prompt)
# Synchronize on virtio-block driver creating the root device
exec_command_and_wait_for_pattern(self,
"while ! (dmesg -c | grep vda:) ; do sleep 1 ; done",
"vda1")
exec_command_and_wait_for_pattern(self, 'mount /dev/vda1 /sysroot', exec_command_and_wait_for_pattern(self, 'mount /dev/vda1 /sysroot',
prompt) prompt)
exec_command_and_wait_for_pattern(self, 'chroot /sysroot', exec_command_and_wait_for_pattern(self, 'chroot /sysroot',
@ -65,10 +70,21 @@ class VirtioBalloonx86(QemuSystemTest):
assert val == UNSET_STATS_VALUE assert val == UNSET_STATS_VALUE
def assert_running_stats(self, then): def assert_running_stats(self, then):
# We told the QEMU to refresh stats every 100ms, but
# there can be a delay between virtio-ballon driver
# being modprobed and seeing the first stats refresh
# Retry a few times for robustness under heavy load
retries = 10
when = 0
while when == 0 and retries:
ret = self.vm.qmp('qom-get', ret = self.vm.qmp('qom-get',
{'path': '/machine/peripheral/balloon', {'path': '/machine/peripheral/balloon',
'property': 'guest-stats'})['return'] 'property': 'guest-stats'})['return']
when = ret.get('last-update') when = ret.get('last-update')
if when == 0:
retries = retries - 1
time.sleep(0.5)
now = time.time() now = time.time()
assert when > then and when < now assert when > then and when < now
@ -94,6 +110,7 @@ class VirtioBalloonx86(QemuSystemTest):
def test_virtio_balloon_stats(self): def test_virtio_balloon_stats(self):
self.set_machine('q35') self.set_machine('q35')
self.require_accelerator("kvm")
kernel_path = self.ASSET_KERNEL.fetch() kernel_path = self.ASSET_KERNEL.fetch()
initrd_path = self.ASSET_INITRD.fetch() initrd_path = self.ASSET_INITRD.fetch()
diskimage_path = self.ASSET_DISKIMAGE.fetch() diskimage_path = self.ASSET_DISKIMAGE.fetch()
@ -106,7 +123,7 @@ class VirtioBalloonx86(QemuSystemTest):
# reset, we can reliably catch the clean stats again in BIOS # reset, we can reliably catch the clean stats again in BIOS
# phase before the guest OS launches # phase before the guest OS launches
self.vm.add_args("-boot", "menu=on") self.vm.add_args("-boot", "menu=on")
self.vm.add_args("-machine", "q35,accel=kvm:tcg") self.vm.add_args("-accel", "kvm")
self.vm.add_args("-device", "virtio-balloon,id=balloon") self.vm.add_args("-device", "virtio-balloon,id=balloon")
self.vm.add_args('-drive', self.vm.add_args('-drive',
f'file={diskimage_path},if=none,id=drv0,snapshot=on') f'file={diskimage_path},if=none,id=drv0,snapshot=on')