Commit graph

324 commits

Author SHA1 Message Date
Thomas Huth
99fb9256b7 tests/functional: Remove unnecessary import statements
pylint complains about these unnecessary import statements,
so let's remove them.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250414145457.261734-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
858640eaee tests/functional: Remove semicolons at the end of lines
Yes, we are all C coders who try to write Python code for testing...
but still, let's better avoid semicolons at the end of the lines
to keep "pylint" happy!

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Message-ID: <20250327201305.996241-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
5c2bae2155 tests/functional: Convert the SMMU test to the functional framework
This test was using cloudinit and a "dnf install" command in the guest
to exercise the NIC with SMMU enabled. Since we don't have the cloudinit
stuff in the functional framework and we should not rely on having access
to external networks (once our ASSETs have been cached), we rather boot
into the initrd first, manually mount the root disk and then use the
check_http_download() function from the functional framework here instead
for testing whether the network works as expected.

Unfortunately, there seems to be a small race when using the files
from Fedora 33: To enter the initrd shell, we have to send a "return"
once. But it does not seem to work if we send it too early. Using a
sleep(0.2) makes it work reliably for me, but to make it even more
unlikely to trigger this situation, let's better limit the Fedora 33
tests to only run with KVM.

Finally, while we're at it, we also add some lines for testing writes
to the hard disk, as we already do it in the test_intel_iommu test.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Message-ID: <20250414113031.151105-14-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
a820caf844 tests/functional: Use the tuxrun kernel for the aarch64 replay test
This way we can do a full boot in record-replay mode and
should get a similar test coverage compared to the old
replay test from tests/avocado/replay_linux.py.

Since the aarch64 test was the last avocado test in the
tests/avocado/replay_linux.py file, we can remove this
file now completely.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250414113031.151105-13-thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
7fecdb0acd tests/functional: Use the tuxrun kernel for the x86 replay test
This way we can do a full boot in record-replay mode and
should get a similar test coverage compared to the old
replay test from tests/avocado/replay_linux.py. Thus remove
the x86 avocado replay_linux test now.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-12-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
f79592f427 tests/functional: Convert the 64-bit big endian Wheezy mips test
Reuse the test function from the 32-bit big endian test to easily
convert the 64-bit big endian Wheezy mips test.

Since this was the last test in tests/avocado/linux_ssh_mips_malta.py,
we can remove this avocado file now, too.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-10-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
8e3461c3a6 tests/functional: Convert the 64-bit little endian Wheezy mips test
Reuse the test function from the 32-bit big endian test to easily
convert the 64-bit little endian Wheezy mips test.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-9-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
689a8b56a6 tests/functional: Convert the 32-bit little endian Wheezy mips test
Reuse the test function from the big endian test to easily
convert the 32-bit little endian Wheezy mips test.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-8-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
42a87f0ce7 tests/functional: Convert the 32-bit big endian Wheezy mips test
The test checks some entries in /proc and the output of some commands ...
we put these checks into exportable functions now so that they can
be reused more easily.

Additionally the linux_ssh_mips_malta.py uses SSH to test the networking
of the guest. Since we don't have a SSH module in the functional
framework yet, let's use the check_http_download() function here instead.

And while we're at it, also switch the NIC to e1000 now to get some more
test coverage, since the "pcnet" device is already tested in the test
test_mips_malta_cpio.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-7-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
0e756f404d tests/functional: Convert the i386 replay avocado test
Since this was the last test in tests/avocado/replay_kernel.py,
we can remove that Avocado file now.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
951ededf12 tests/functional: Convert reverse_debugging tests to the functional framework
These tests are using the gdb-related library functions from the
Avocado framework which we don't have in the functional framework
yet. So for the time being, keep those imports and skip the test
if the Avocado framework is not installed on the host.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Thomas Huth
bc65ae6961 tests/functional: Move the check for the parameters from avocado to functional
test_x86_64_pc in tests/avocado/boot_linux_console.py only checks
whether the kernel parameters have correctly been passed to the
kernel in the guest by looking for them in the console output of the
guest. Let's move that to the functional test framework now, but
instead of doing it in a separate test, let's do it for all tuxrun
tests instead, so it is done automatically for all targets that have
a tuxrun test.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Cornelia Huck
4e3823c68c tests/functional/test_vnc: skip test if no crypto backend available
The test_change_password test will fail if no cryptographic backend is
available (e.g. if QEMU was built on a system with no cryptographic
library development packages installed); just skip the test in that
case.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250414093732.220498-1-cohuck@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-04-23 07:51:25 +02:00
Pierrick Bouvier
8bdd3a0308 tests/functional/test_aarch64_replay: reenable on macos
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250410225550.46807-3-pierrick.bouvier@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2025-04-14 11:03:16 -04:00
Philippe Mathieu-Daudé
bd20bc46fe tests/functional: Skip aarch64_replay test on macOS
As of v10.0.0-rc2 this test is still failing on macos:

  $ make check-functional-aarch64 V=1
  ...
  ERROR:../../replay/replay-internal.c:235:replay_mutex_unlock: assertion failed: (replay_mutex_locked())
  Bail out! ERROR:../../replay/replay-internal.c:235:replay_mutex_unlock: assertion failed: (replay_mutex_locked())

This is tracked as https://gitlab.com/qemu-project/qemu/-/issues/2907

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-4-philmd@linaro.org>
2025-04-03 16:11:09 -07:00
Philippe Mathieu-Daudé
00f119f4c4 tests/functional: Add a decorator for skipping tests on particular OS
Since tests might be failing on some operating systems,
introduce the skipIfOperatingSystem() decorator.

Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-3-philmd@linaro.org>
2025-04-03 16:11:09 -07:00
Pierrick Bouvier
4412d71382 tests/functional/test_aarch64_rme_virt: fix sporadic failure
This test was randomly failing on our CI, and on dev machines,
especially with QEMU debug builds.

>From the information collected, it's related to an implementation choice
in edk2 QEMU virt support. The workaround is to disable KASLR, to avoid
accessing protected memory.
Note: this is *not* needed for the similar test_aarch64_rme_sbsaref.

More information is available on the associated GitLab issue.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2823
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250328183816.2687925-1-pierrick.bouvier@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-2-philmd@linaro.org>
2025-04-03 16:11:09 -07:00
Thomas Huth
5a8d024f8d tests/functional/test_vnc: Skip test if VNC support is not available
These tests currently fail if VNC support has not been compiled into
the QEMU binary. Let's add some checks to skip the tests in that
case instead.

Message-ID: <20250325064715.278876-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Thomas Huth
c3612d0b6a tests/functional/test_aarch64_virt_gpu: Skip if "dbus" display isn't available
This test currently fails if the "dbus" display has not been compiled
into the binary (which can happen when CFI has been enabled, for example).
Check for the error message to skip the test in that case.

While we're at it, also make sure that this test is covered in the
right section in the MAINTAINERS file.

Message-ID: <20250325061609.272847-1-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Thomas Huth
dba0752f2c tests/functional: Add missing require_netdev('user') statements
A bunch of tests are using "-netdev user" but fail to check
for the availability of SLIRP in the binary, so these tests
fail if QEMU has been configured with "--disable-slirp"
(most of the tests are disabled by default with a decorator,
that's likely why nobody noticed this problem yet). Add the
missing self.require_netdev('user') statements to skip the
tests if SLIRP is not available.

Message-ID: <20250324123450.111307-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Peter Maydell
f0a6b3ec6d tests/functional/meson.build: Bump arm_aspeed_bletchley timeout
The arm_aspeed_bletchley timeout takes more than its
current 120s timeout on a debug Rust config:

$ time (cd build/rust; PYTHONPATH=../../python:../../tests/functional QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3 ../../tests/functional/test_arm_aspeed_bletchley.py)
TAP version 13
ok 1 test_arm_aspeed_bletchley.BletchleyMachine.test_arm_ast2600_bletchley_openbmc
1..1

real    2m15.536s
user    3m21.444s
sys     0m11.558s

Bump it up to 480s, same as arm_aspeed_rainier.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250321155806.1888867-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Thomas Huth
3b9731020d tests/functional/test_ppc64_replay: Mark the e500 test as flaky
The test_ppc64_e500 occasionally fails (it just hangs and finally times
out). The same issue could also be reproduced with the former Avocado
test already (especially if the host system is under heavy load), so
it's not a new regression. It's very likely the issue that has been
filed at https://gitlab.com/qemu-project/qemu/-/issues/2523 instead (e.g.
I could not reproduce the issue in older commits before commit 578912ad),
so use this URL for the reasoning in the the decorator.

Message-ID: <20250320065012.309520-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Thomas Huth
f700abbbeb tests/functional/test_migration: Use "ncat" instead of "nc" in the exec test
"nc" can either be GNU netcat, OpenBSD netcat or NMap ncat. At
least GNU netcat currently does not work with this test anymore,
though the comment in the test says otherwise. GNU netcat seems
to be quite unmaintained nowadays, according to its website
(https://netcat.sourceforge.net/), the last public release is from
2004, so we should rather avoid that binary.
In our CI, we are only using "ncat" in the containers (it's the only
flavor that lcitool supports), thus to avoid silent regressions with
the other netcats, let's limit this test to "ncat" only now.

Reported-by: Matheus Tavares Bernardino <quic_mathbern@quicinc.com>
Message-ID: <20250319105617.133191-1-thuth@redhat.com>
Reviewed-by: Matheus Tavares Bernardino <quic_mathbern@quicinc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-19 13:28:34 +01:00
Thomas Huth
1ddd4d3f6e tests/functional/test_x86_64_kvm_xen: Remove avocado tags
They have been forgotten to be removed when converting the
test to the functional framework. Since they are of no use
anymore, let's remove them now.

Message-ID: <20250318171530.94966-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-19 07:43:51 +01:00
Daniel P. Berrangé
23686dfb76 tests/functional: remove all class level fields
A number of fields are set at the class level on QemuBaseTest, even
though the exact same named field is then set at the object level
later in most cases.

The 'self.logger' initialization in ACPI bits test needs to be removed
since 'self.log' won't exist at that point in the flow. It already
initialized 'self.logger' later in the setUp() method, so the __init__
method was redundant.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250317124300.84266-1-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-19 07:41:32 +01:00
Niek Linnenbank
6d19d09547 tests/functional/test_arm_orangepi: rename test class to 'OrangePiMachine'
The test class in this file contains all functional test cases
for testing the Orange Pi PC board. It should be given a name
matching the Qemu machine it covers.

This commit sets the test class name to 'OrangePiMachine'.

Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Fixes: 380f7268b7 ("tests/functional: Convert the OrangePi tests to the  functional framework")
Message-ID: <20250316210232.46298-1-nieklinnenbank@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-19 07:41:32 +01:00
Stefan Hajnoczi
28ea66f6f9 * Various fixes for functional tests
* Fix the name of the "configs" directory in the documentation
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmfSjagRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbWBmA//RhAHuF/fTmQagBsZPETXjU1g8ifw9aqm
 WPZcQEXyQFlqYYQZmtV7dk3aTGEw4kBDmm+SKTSQz1yUcBGptMl8xuWaxgdpcOw0
 Bqt+lYNgwGL9/OocCdNolU3+aVbETljr5l+rzbnwsTVIqGk63Qhmtwdupb8h1nfY
 4vCXU+sY3BkvBF8HbV6Wb1aPtqC+iH/Ln8+yoKkC8UePD623dK58SsOVrhUQDfFr
 U/HUy4BZlHFCfGGmDVGBjHdEbOzQkLQ9N3ilsNSWcF87RPkWPft+qLs4RjDFW+oT
 oksXEFHcr8XQO03fwHBNTyv+NUfnrvDY8V+gl6C9ItQr58SZzse57caZKWrYppZ3
 l5iHoaLMV3juZFDNXNHkWHuveXi05+0V0UbZihzBeC4+zjNRyh3e1GuDoh5VoG8o
 XIb55RxU8eBG2/ulHZ71eAYrGpxO+tDdsdnak1coPFsU8HrC9QzRfywiAZe1Wwmx
 5t5AHbZ7RdnxgStU1lWTUT2IDVSini4DKevt/FzhKkv1aD8NbhI/ooGDC0zbS6SU
 XK6PP2G5a5OnjQ904oRCQbnhrxFa5qNfryylvvreT2bVgX0BiE4pJ9JXdgQOMYlP
 kZERZZQcv3y6VVavAT67yeNKQpyb4HSHdTDQ2irgXP1UwHRpwLpKdqB1UhzNJ8m8
 k0faA8RXir4=
 =VtGZ
 -----END PGP SIGNATURE-----

Merge tag 'pull-request-2025-03-13' of https://gitlab.com/thuth/qemu into staging

* Various fixes for functional tests
* Fix the name of the "configs" directory in the documentation

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmfSjagRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbWBmA//RhAHuF/fTmQagBsZPETXjU1g8ifw9aqm
# WPZcQEXyQFlqYYQZmtV7dk3aTGEw4kBDmm+SKTSQz1yUcBGptMl8xuWaxgdpcOw0
# Bqt+lYNgwGL9/OocCdNolU3+aVbETljr5l+rzbnwsTVIqGk63Qhmtwdupb8h1nfY
# 4vCXU+sY3BkvBF8HbV6Wb1aPtqC+iH/Ln8+yoKkC8UePD623dK58SsOVrhUQDfFr
# U/HUy4BZlHFCfGGmDVGBjHdEbOzQkLQ9N3ilsNSWcF87RPkWPft+qLs4RjDFW+oT
# oksXEFHcr8XQO03fwHBNTyv+NUfnrvDY8V+gl6C9ItQr58SZzse57caZKWrYppZ3
# l5iHoaLMV3juZFDNXNHkWHuveXi05+0V0UbZihzBeC4+zjNRyh3e1GuDoh5VoG8o
# XIb55RxU8eBG2/ulHZ71eAYrGpxO+tDdsdnak1coPFsU8HrC9QzRfywiAZe1Wwmx
# 5t5AHbZ7RdnxgStU1lWTUT2IDVSini4DKevt/FzhKkv1aD8NbhI/ooGDC0zbS6SU
# XK6PP2G5a5OnjQ904oRCQbnhrxFa5qNfryylvvreT2bVgX0BiE4pJ9JXdgQOMYlP
# kZERZZQcv3y6VVavAT67yeNKQpyb4HSHdTDQ2irgXP1UwHRpwLpKdqB1UhzNJ8m8
# k0faA8RXir4=
# =VtGZ
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 13 Mar 2025 15:47:52 HKT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2025-03-13' of https://gitlab.com/thuth/qemu:
  tests/functional: skip vulkan test if missing vulkaninfo
  tests/functional/asset: Add AssetError exception class
  tests/functional/asset: Verify downloaded size
  tests/functional/asset: Fail assert fetch when retries are exceeded
  docs/system: Fix the information on how to run certain functional tests
  tests/functional: Bump up arm_replay timeout
  tests/functional: Require 'user' netdev for ppc64 e500 test
  docs: Rename default-configs to configs

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2025-03-14 09:30:53 +08:00
Alex Bennée
b3c03666fb tests/functional: skip vulkan test if missing vulkaninfo
I could have sworn I had this is a previous iteration of the patches
but I guess it got lost in a re-base. As we are going to call
vulkaninfo to probe for "bad" drivers we need to skip if the binary
isn't available.

Fixes: 9f7e493d11 (tests/functional: skip vulkan tests with nVidia)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250312190314.1632357-1-alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-13 08:18:26 +01:00
Nicholas Piggin
28adad0a4d tests/functional/asset: Add AssetError exception class
Assets are uniquely identified by human-readable-ish url, so make an
AssetError exception class that prints url with error message.

A property 'transient' is used to capture whether the client may retry
or try again later, or if it is a serious and likely permanent error.
This is used to retain the existing behaviour of treating HTTP errors
other than 404 as 'transient' and not causing precache step to fail.
Additionally, partial-downloads and stale asset caches that fail to
resolve after the retry limit are now treated as transient and do not
cause precache step to fail.

For background: The NetBSD archive is, at the time of writing, failing
with short transfer. Retrying the fetch at that position (as wget does)
results in a "503 backend unavailable" error. We would like to get that
error code directly, but I have not found a way to do that with urllib,
so treating the short-copy as a transient failure covers that case (and
seems like a reasonable way to handle it in general).

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-4-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:20:50 +01:00
Nicholas Piggin
7524e1b336 tests/functional/asset: Verify downloaded size
If the server provides a Content-Length header, use that to verify the
size of the downloaded file. This catches cases where the connection
terminates early, and gives the opportunity to retry. Without this, the
checksum will likely mismatch and fail without retry.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-3-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:20:50 +01:00
Nicholas Piggin
a5e8299d1a tests/functional/asset: Fail assert fetch when retries are exceeded
Currently the fetch code does not fail gracefully when retry limit is
exceeded, it just falls through the loop with no file, which ends up
hitting other errors.

Add a check for non-existing file, which indicates the retry limit was
exceeded.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-2-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:20:50 +01:00
Peter Maydell
8c63f9aa3f tests/functional: Bump up arm_replay timeout
On my machine the arm_replay test takes over 2 minutes to run
in a config with Rust enabled and debug enabled:

$ time (cd build/rust ; PYTHONPATH=../../python:../../tests/functional
QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3
../../tests/functional/test_arm_replay.py)
TAP version 13
ok 1 test_arm_replay.ArmReplay.test_cubieboard
ok 2 test_arm_replay.ArmReplay.test_vexpressa9
ok 3 test_arm_replay.ArmReplay.test_virt
1..3

real    2m16.564s
user    2m13.461s
sys     0m3.523s

Bump up the timeout to 4 minutes.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250310102830.3752440-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:15:44 +01:00
Cédric Le Goater
533b33d04b tests/functional: Require 'user' netdev for ppc64 e500 test
When commit 72cdd672e1 extended the ppc64 e500 test to add network
support, it forgot to require the 'user' netdev backend. Fix that.

Fixes: 72cdd672e1 ("tests/functional: Replace the ppc64 e500 advent calendar test")
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250308071328.193694-1-clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:15:30 +01:00
Cédric Le Goater
e5029e28f0 ppc/ppc405: Remove tests
Since we are about to remove all support for PPC 405, start by
removing the tests referring to the ref405ep machine.

Link: https://lore.kernel.org/qemu-devel/20250110141800.1587589-2-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250204080649.836155-2-clg@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2025-03-11 22:40:47 +10:00
Pierrick Bouvier
2a8e8544b2 tests/functional: add boot error detection for RME tests
It was identified that those tests randomly fail with a synchronous
exception at boot (reported by EDK2).
While we solve this problem, report failure immediately so tests don't
timeout in CI.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20250303185745.2504842-1-pierrick.bouvier@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-20-alex.bennee@linaro.org>
2025-03-10 10:30:01 +00:00
Philippe Mathieu-Daudé
ed557cc544 tests/functional: Allow running TCG plugins tests on non-Linux/BSD hosts
Not all platforms use the '.so' suffix for shared libraries,
which is how plugins are built. Use the recently introduced
dso_suffix() helper to get the proper host suffix.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2804
Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250220080215.49165-4-philmd@linaro.org>
[AJB: moved plugin_file into testcase.py]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-11-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Philippe Mathieu-Daudé
3a7b9054b8 tests/functional: Introduce the dso_suffix() helper
Introduce a helper to get the default shared library
suffix used on the host.

Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250220080215.49165-3-philmd@linaro.org>
[AJB: dropped whitespace cmd.py damage]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-10-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Pierrick Bouvier
892b06c40e plugins: add explicit dependency in functional tests
./tests/functional/test_aarch64_tcg_plugins.py needs to have plugin
libinsn built. However, it's not listed as a dependency, so meson can't
know it needs to be built.

Thus, we keep track of all plugins, and add them as an explicit
dependency.

Fixes: 4c134d07b9 ("tests: add a new set of tests to exercise plugins")
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-9-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
d69178d370 tests/functional: update the aarch64_virg_gpu images
Update to the most recent aarch64_virt_gpu image. The principle
differences are:

  - target a v8.0 baseline CPU
  - latest vkmark (2025.1)
  - actually uses the rootfs (previously was initrd)
  - rootfs includes more testing tools for interactive use

See README.md in https://fileserver.linaro.org/s/ce5jXBFinPxtEdx for
details about the image creation and the buildroot config.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-8-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
8233f4f26d tests/functional: expand tests to cover virgl
Add two more test modes using glmark2-wayland to exercise the OpenGL
pass-through modes with virgl. Virgl can run with or without the
hostmem blob support. To avoid repeating ourselves too much we make
the initial pass a simple --validate pass.

We might want to eventually add more directed tests and individual
features later on but the glmark/vkmark tests are a good general
smoke test for accelerated 3D.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-7-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
9f7e493d11 tests/functional: skip vulkan tests with nVidia
While running the new GPU tests it was noted that the proprietary
nVidia driver barfed when run under the sanitiser:

  2025-02-20 11:13:08,226: [11:13:07.782] Output 'headless' attempts
  EOTF mode SDR and colorimetry mode default.
  2025-02-20 11:13:08,227: [11:13:07.784] Output 'headless' using color
  profile: stock sRGB color profile

  and that's the last thing it outputs.

  The sanitizer reports that when the framework sends the SIGTERM
  because of the timeout we get a write to a NULL pointer (but
  interesting not this time in an atexit callback):

  UndefinedBehaviorSanitizer:DEADLYSIGNAL
  ==471863==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address
  0x000000000000 (pc 0x7a18ceaafe80 bp 0x000000000000 sp 0x7ffe8e3ff6d0
  T471863)
  ==471863==The signal is caused by a WRITE memory access.
  ==471863==Hint: address points to the zero page.
      #0 0x7a18ceaafe80
  (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x16afe80)
  (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
      #1 0x7a18ce9e72c0
  (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15e72c0)
  (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
      #2 0x7a18ce9f11bb
  (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15f11bb)
  (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
      #3 0x7a18ce6dc9d1
  (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x12dc9d1)
  (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
      #4 0x7a18e7d15326 in vrend_renderer_create_fence
  /usr/src/virglrenderer-1.0.0-1ubuntu2/obj-x86_64-linux-gnu/../src/vrend_renderer.c:10883:26
      #5 0x55bfb6621871 in virtio_gpu_virgl_process_cmd

The #dri-devel channel confirmed:

  <digetx> stsquad: nv driver is known to not work with venus, don't use
      it for testing

So lets skip running the test to avoid known failures. As we now use
vulkaninfo to probe we also need to handle the case where there is no
Vulkan driver configured for the hardware.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Cc: Dmitry Osipenko <dmitry.osipenko@collabora.com>
[AJB: also skip if vulkaninfo can't find environment]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-6-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
4e4e6986b6 tests/functional: bail early if vkmark hangs
The message:

  MESA-VIRTIO: debug: stuck in fence wait with iter at %d

Seems to occur more often on debug builds. Rather than waiting for our
long timeout to hit we might as well bail as soon as we see the
message.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-5-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
94634fbc66 tests/functional: ensure we have a GPU device for tests
It's possible to build QEMU without support for the GL enabled GPU
devices and we can catch that earlier with an explicit check.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-4-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
868c7703e0 tests/functional: factor out common code in gpu test
In preparation for handling more tests split out the common machine
setup details from the test specific stuff and add a helper for
launching the weston test. Instead of searching for "vkmark score" we
set a custom PS1 and wait for a successful completion. This ensures we
capture the score in the console log which otherwise wouldn't log
anything.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-3-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
9a72282560 tests/functional: move aarch64 GPU test into own file
I want to expand the number of tests to cover a wide range of
configurations. That starts with splitting off from the normal virt
test from which it doesn't really share much code. We can also reduce
the timeout of the original virt test now it is now longer burdened
with testing the GPU.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-2-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Jamin Lin
2cab06e930 tests/functional/aspeed: Add test case for AST2700 A1
Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250307035945.3698802-29-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:53 +01:00
Jamin Lin
c77ec95a4b tests/functional/aspeed: Update test ASPEED SDK v09.05
In ASPEED SDK v09.05, the naming convention for pre-built images has been
updated. The pre-built image for AST2700 A0 has been renamed to
ast2700-a0-default, while ast2700-default is now used for AST2700 A1.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250307035945.3698802-28-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:53 +01:00
Jamin Lin
8241d6f8d5 tests/functional/aspeed: Update temperature hwmon path
Modified the temperature hwmon path to use a wildcard to handle different SDK
versions: "cat /sys/bus/i2c/devices/1-004d/hwmon/hwmon*/temp1_input".

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250307035945.3698802-27-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:53 +01:00
Jamin Lin
827eecb0e8 tests/functional/aspeed: Introduce start_ast2700_test API
Added a new method "start_ast2700_test" to the "AST2x00MachineSDK" class and
this method centralizes the logic for starting the AST2700 test, making it
reusable for different test cases.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250307035945.3698802-26-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:53 +01:00
Cédric Le Goater
b91a1d3110 tests/functional: Introduce a bletchley machine test
Use do_test_arm_aspeed_openbmc() to run the latest OpenBMC firmware
build of the bletchley BMC.

Reviewed-by: Patrick Williams <patrick@stwcx.xyz>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250129071820.1258133-6-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:52 +01:00