It is possible nowadays to compile QEMU without pixman support - in that
case the screendump command is not available and the related tests fail.
Thus skip these tests if the screendump command could not be executed.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250325081713.283490-2-thuth@redhat.com>
The s390x tuxrun test lacks the call to self.set_machine(), so this
test is currently failing in case the 's390-ccw-virtio' machine has
not been compiled into the binary. Add the check now to fix it.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250424090640.664217-1-thuth@redhat.com>
qemu-system-hppa shuts down automatically when the BIOS is
unable to boot from any device. So this test currently fails
occasionally when QEMU already quit, but the test still
expected it to be around (e.g. to shut it down cleanly).
Adding a "-no-shutdown" seems to make it reliable.
While we're at it, also remove the stray "self.machine" in
there that does not have any purpose.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20250508180918.228757-1-thuth@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Introduce a functional test which boots Debian 12 on the imx8mp-evk board. Since
the root filesystem resides on an SD card, the test also verifies the basic
operation of the USDHC.
Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20250409202630.19667-1-shentey@gmail.com
[PMM: added extra blank line as suggested by thuth;
set timeout to 240s]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Introduce a new test suite for ast2700fc machine.
Rename the original test_aarch64_aspeed.py to
test_aarch64_aspeed_ast2700.py.
Signed-off-by: Steven Lee <steven_lee@aspeedtech.com>
Change-Id: I3855f55c9f6e5cca1270c179445f549f8d81f36c
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250505030618.3612042-1-steven_lee@aspeedtech.com
[ clg: Added new tests in meson.build ]
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Add the AST2700 functional test to boot using the vbootrom image
instead of manually loading boot components with -device loader.
The boot ROM binary is now passed via the
-bios option, using the image located in pc-bios/ast27x0_bootrom.bin.
Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250424075135.3715128-5-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Extracted repeated boot and login steps into a new helper function.
No change in functional behavior.
Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250423072350.541742-10-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Move the I2C test case into a common helper function (do_ast2700_i2c_test) so it
can be reused across multiple AST2700-based test cases. This reduces duplication
and improves maintainability.
Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Nabih Estefan <nabihestefan@google.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250423072350.541742-8-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
The test_ppc64_linux_smt_boot function lacks the set_machine('pseries'),
so this test is currently failing in case the 'pseries' machine has not
been compiled into the binary. Add the check now to fix it.
Message-ID: <20250424085426.663377-1-thuth@redhat.com>
Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
pylint complains about these unnecessary import statements,
so let's remove them.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250414145457.261734-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Yes, we are all C coders who try to write Python code for testing...
but still, let's better avoid semicolons at the end of the lines
to keep "pylint" happy!
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Message-ID: <20250327201305.996241-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This test was using cloudinit and a "dnf install" command in the guest
to exercise the NIC with SMMU enabled. Since we don't have the cloudinit
stuff in the functional framework and we should not rely on having access
to external networks (once our ASSETs have been cached), we rather boot
into the initrd first, manually mount the root disk and then use the
check_http_download() function from the functional framework here instead
for testing whether the network works as expected.
Unfortunately, there seems to be a small race when using the files
from Fedora 33: To enter the initrd shell, we have to send a "return"
once. But it does not seem to work if we send it too early. Using a
sleep(0.2) makes it work reliably for me, but to make it even more
unlikely to trigger this situation, let's better limit the Fedora 33
tests to only run with KVM.
Finally, while we're at it, we also add some lines for testing writes
to the hard disk, as we already do it in the test_intel_iommu test.
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Message-ID: <20250414113031.151105-14-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This way we can do a full boot in record-replay mode and
should get a similar test coverage compared to the old
replay test from tests/avocado/replay_linux.py.
Since the aarch64 test was the last avocado test in the
tests/avocado/replay_linux.py file, we can remove this
file now completely.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250414113031.151105-13-thuth@redhat.com>
This way we can do a full boot in record-replay mode and
should get a similar test coverage compared to the old
replay test from tests/avocado/replay_linux.py. Thus remove
the x86 avocado replay_linux test now.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-12-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reuse the test function from the 32-bit big endian test to easily
convert the 64-bit big endian Wheezy mips test.
Since this was the last test in tests/avocado/linux_ssh_mips_malta.py,
we can remove this avocado file now, too.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-10-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reuse the test function from the 32-bit big endian test to easily
convert the 64-bit little endian Wheezy mips test.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-9-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reuse the test function from the big endian test to easily
convert the 32-bit little endian Wheezy mips test.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-8-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The test checks some entries in /proc and the output of some commands ...
we put these checks into exportable functions now so that they can
be reused more easily.
Additionally the linux_ssh_mips_malta.py uses SSH to test the networking
of the guest. Since we don't have a SSH module in the functional
framework yet, let's use the check_http_download() function here instead.
And while we're at it, also switch the NIC to e1000 now to get some more
test coverage, since the "pcnet" device is already tested in the test
test_mips_malta_cpio.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-7-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Since this was the last test in tests/avocado/replay_kernel.py,
we can remove that Avocado file now.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
These tests are using the gdb-related library functions from the
Avocado framework which we don't have in the functional framework
yet. So for the time being, keep those imports and skip the test
if the Avocado framework is not installed on the host.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
test_x86_64_pc in tests/avocado/boot_linux_console.py only checks
whether the kernel parameters have correctly been passed to the
kernel in the guest by looking for them in the console output of the
guest. Let's move that to the functional test framework now, but
instead of doing it in a separate test, let's do it for all tuxrun
tests instead, so it is done automatically for all targets that have
a tuxrun test.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The test_change_password test will fail if no cryptographic backend is
available (e.g. if QEMU was built on a system with no cryptographic
library development packages installed); just skip the test in that
case.
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250414093732.220498-1-cohuck@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
As of v10.0.0-rc2 this test is still failing on macos:
$ make check-functional-aarch64 V=1
...
ERROR:../../replay/replay-internal.c:235:replay_mutex_unlock: assertion failed: (replay_mutex_locked())
Bail out! ERROR:../../replay/replay-internal.c:235:replay_mutex_unlock: assertion failed: (replay_mutex_locked())
This is tracked as https://gitlab.com/qemu-project/qemu/-/issues/2907
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-4-philmd@linaro.org>
Since tests might be failing on some operating systems,
introduce the skipIfOperatingSystem() decorator.
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-3-philmd@linaro.org>
This test was randomly failing on our CI, and on dev machines,
especially with QEMU debug builds.
>From the information collected, it's related to an implementation choice
in edk2 QEMU virt support. The workaround is to disable KASLR, to avoid
accessing protected memory.
Note: this is *not* needed for the similar test_aarch64_rme_sbsaref.
More information is available on the associated GitLab issue.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2823
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250328183816.2687925-1-pierrick.bouvier@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-2-philmd@linaro.org>
These tests currently fail if VNC support has not been compiled into
the QEMU binary. Let's add some checks to skip the tests in that
case instead.
Message-ID: <20250325064715.278876-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This test currently fails if the "dbus" display has not been compiled
into the binary (which can happen when CFI has been enabled, for example).
Check for the error message to skip the test in that case.
While we're at it, also make sure that this test is covered in the
right section in the MAINTAINERS file.
Message-ID: <20250325061609.272847-1-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
A bunch of tests are using "-netdev user" but fail to check
for the availability of SLIRP in the binary, so these tests
fail if QEMU has been configured with "--disable-slirp"
(most of the tests are disabled by default with a decorator,
that's likely why nobody noticed this problem yet). Add the
missing self.require_netdev('user') statements to skip the
tests if SLIRP is not available.
Message-ID: <20250324123450.111307-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The arm_aspeed_bletchley timeout takes more than its
current 120s timeout on a debug Rust config:
$ time (cd build/rust; PYTHONPATH=../../python:../../tests/functional QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3 ../../tests/functional/test_arm_aspeed_bletchley.py)
TAP version 13
ok 1 test_arm_aspeed_bletchley.BletchleyMachine.test_arm_ast2600_bletchley_openbmc
1..1
real 2m15.536s
user 3m21.444s
sys 0m11.558s
Bump it up to 480s, same as arm_aspeed_rainier.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250321155806.1888867-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The test_ppc64_e500 occasionally fails (it just hangs and finally times
out). The same issue could also be reproduced with the former Avocado
test already (especially if the host system is under heavy load), so
it's not a new regression. It's very likely the issue that has been
filed at https://gitlab.com/qemu-project/qemu/-/issues/2523 instead (e.g.
I could not reproduce the issue in older commits before commit 578912ad),
so use this URL for the reasoning in the the decorator.
Message-ID: <20250320065012.309520-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
"nc" can either be GNU netcat, OpenBSD netcat or NMap ncat. At
least GNU netcat currently does not work with this test anymore,
though the comment in the test says otherwise. GNU netcat seems
to be quite unmaintained nowadays, according to its website
(https://netcat.sourceforge.net/), the last public release is from
2004, so we should rather avoid that binary.
In our CI, we are only using "ncat" in the containers (it's the only
flavor that lcitool supports), thus to avoid silent regressions with
the other netcats, let's limit this test to "ncat" only now.
Reported-by: Matheus Tavares Bernardino <quic_mathbern@quicinc.com>
Message-ID: <20250319105617.133191-1-thuth@redhat.com>
Reviewed-by: Matheus Tavares Bernardino <quic_mathbern@quicinc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Thomas Huth <thuth@redhat.com>
They have been forgotten to be removed when converting the
test to the functional framework. Since they are of no use
anymore, let's remove them now.
Message-ID: <20250318171530.94966-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
A number of fields are set at the class level on QemuBaseTest, even
though the exact same named field is then set at the object level
later in most cases.
The 'self.logger' initialization in ACPI bits test needs to be removed
since 'self.log' won't exist at that point in the flow. It already
initialized 'self.logger' later in the setUp() method, so the __init__
method was redundant.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250317124300.84266-1-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The test class in this file contains all functional test cases
for testing the Orange Pi PC board. It should be given a name
matching the Qemu machine it covers.
This commit sets the test class name to 'OrangePiMachine'.
Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Fixes: 380f7268b7 ("tests/functional: Convert the OrangePi tests to the functional framework")
Message-ID: <20250316210232.46298-1-nieklinnenbank@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
I could have sworn I had this is a previous iteration of the patches
but I guess it got lost in a re-base. As we are going to call
vulkaninfo to probe for "bad" drivers we need to skip if the binary
isn't available.
Fixes: 9f7e493d11 (tests/functional: skip vulkan tests with nVidia)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250312190314.1632357-1-alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Assets are uniquely identified by human-readable-ish url, so make an
AssetError exception class that prints url with error message.
A property 'transient' is used to capture whether the client may retry
or try again later, or if it is a serious and likely permanent error.
This is used to retain the existing behaviour of treating HTTP errors
other than 404 as 'transient' and not causing precache step to fail.
Additionally, partial-downloads and stale asset caches that fail to
resolve after the retry limit are now treated as transient and do not
cause precache step to fail.
For background: The NetBSD archive is, at the time of writing, failing
with short transfer. Retrying the fetch at that position (as wget does)
results in a "503 backend unavailable" error. We would like to get that
error code directly, but I have not found a way to do that with urllib,
so treating the short-copy as a transient failure covers that case (and
seems like a reasonable way to handle it in general).
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-4-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
If the server provides a Content-Length header, use that to verify the
size of the downloaded file. This catches cases where the connection
terminates early, and gives the opportunity to retry. Without this, the
checksum will likely mismatch and fail without retry.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-3-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Currently the fetch code does not fail gracefully when retry limit is
exceeded, it just falls through the loop with no file, which ends up
hitting other errors.
Add a check for non-existing file, which indicates the retry limit was
exceeded.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-2-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
On my machine the arm_replay test takes over 2 minutes to run
in a config with Rust enabled and debug enabled:
$ time (cd build/rust ; PYTHONPATH=../../python:../../tests/functional
QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3
../../tests/functional/test_arm_replay.py)
TAP version 13
ok 1 test_arm_replay.ArmReplay.test_cubieboard
ok 2 test_arm_replay.ArmReplay.test_vexpressa9
ok 3 test_arm_replay.ArmReplay.test_virt
1..3
real 2m16.564s
user 2m13.461s
sys 0m3.523s
Bump up the timeout to 4 minutes.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250310102830.3752440-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
When commit 72cdd672e1 extended the ppc64 e500 test to add network
support, it forgot to require the 'user' netdev backend. Fix that.
Fixes: 72cdd672e1 ("tests/functional: Replace the ppc64 e500 advent calendar test")
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250308071328.193694-1-clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Since we are about to remove all support for PPC 405, start by
removing the tests referring to the ref405ep machine.
Link: https://lore.kernel.org/qemu-devel/20250110141800.1587589-2-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250204080649.836155-2-clg@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
It was identified that those tests randomly fail with a synchronous
exception at boot (reported by EDK2).
While we solve this problem, report failure immediately so tests don't
timeout in CI.
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20250303185745.2504842-1-pierrick.bouvier@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-20-alex.bennee@linaro.org>
Not all platforms use the '.so' suffix for shared libraries,
which is how plugins are built. Use the recently introduced
dso_suffix() helper to get the proper host suffix.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2804
Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250220080215.49165-4-philmd@linaro.org>
[AJB: moved plugin_file into testcase.py]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-11-alex.bennee@linaro.org>
