All usage has been replaced by direct 'subprocess' helpers.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-30-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The 'run_cmd' helper is re-implementing a convenient helper that
already exists in the form of the 'run' and 'check_call' methods
provided by 'subprocess'.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-29-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Now that all tests are converted over to the higher level wrapper
functions, the back compat imports from utils.py are redundant.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-28-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Replace use of lzma_uncompress and gzip_uncompress with the
new uncompress helper.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-27-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This helper wrappers utils.uncompress, forcing the use of the scratch
directory, to ensure any uncompressed files are cleaned at test
termination.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-26-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
There are many types of compression that the tests deal with, and
it makes sense to have a single helper 'uncompress' that can deal
with all.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-25-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Replace use of utils.archive_extract and extract_from_deb with the
new archive_extract helper.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-24-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This helper wrappers archive.archive_extract, forcing the use of the
scratch directory, to ensure any extracted files are cleaned at test
termination. If a specific member is requested, then the path to the
extracted file is also returned.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-23-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
There are many types of archives that the tests deal with. Provide
a generalized 'archive_extract' that can detect the format and
delegate to the appropriate helper for extraction. This ensures
that all archive extraction code follows the same design pattern.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-22-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Currently cpio_extract differs from tar_extract/zip_extract
in that it only allows a file-like object as input. Adapt it
to also support filenames.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-21-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This mirrors the existing archive_extract, cpio_extract and zip_extract
helpers
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-20-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This mirrors the existing archive_extract and cpio_extract helpers
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-19-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
More uncompress related code will be added shortly, so having a
separate file makes more sense.
The utils.py imports the functions from archive.py, so that
existing callers don't need to be modified. This avoids
redundant code churn until later in the series when all
calls will be adapted for other reasons.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-18-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
More archive related code will be added shortly, so having a
separate file makes more sense.
The utils.py imports the functions from archive.py, so that
existing callers don't need to be modified. This avoids
redundant code churn until later in the series when all
calls will be adapted for other reasons.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-17-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Everything in the scratch directory is automatically purged. Calling
'rmtree' again breaks the ability to optionally preserve the scratch
directory contents.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-16-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Replace any instances of
os.path.join(self.workdir, ".../...")
self.workdir + "/.../..."
with
self.scratch_file("...", "...")
which is more compact and portable
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-15-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This removes direct path manipulation to figure out the source dir
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-14-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This removes direct access of the 'BUILD_DIR' variable.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-13-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This removes direct access of the 'self.logdir' variable.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-12-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add helper methods that construct paths for
* log files - to be preserved at the end of a test
* scratch files - to be purged at the end of a test
* build files - anything relative to the build root
* data files - anything relative to the functional test source root
* socket files - a short temporary dir to avoid UNIX socket limits
These are to be used instead of direct access to the self.workdir,
or self.logdir variables, or any other place where paths are built
manually.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-11-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The 'which' helper is simpler, not depending on the external 'which'
binary, and is sufficient for test needs.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-10-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This ensures consistency of behaviour across all the tests, and requires
that we provide gitlab bug links when marking a test to be skipped due
to unreliability.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-9-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reduce repeated boilerplate with some helper decorators:
@skipIfNotPlatform("x86_64", "aarch64")
=> Skip unless the build host platform matches
@skipIfMissingCommands("mkisofs", "losetup")
=> Skips unless all listed commands are found in $PATH
@skipIfMissingImports("numpy", "cv2")
=> Skips unless all listed modules can be imported
@skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/NNN")
=> Skips unless env var requests flaky tests with the
reason documented in the referenced gitlab bug
@skipBigData
=> Skips unless env var permits tests creating big data files
@skipUntrustedTest
=> Skips unless env var permits tests which are potentially
dangerous to the host
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-8-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Platforms we target have new enough tesseract that it suffices to merely
check if the binary exists.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20241217155953.3950506-7-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The 'access' check implies the file exists.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-6-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Put the 'which' function into shared code.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Allow an Asset object to be used in place of a filename but
making its string representation resolve to the cache file
path.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20241217155953.3950506-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Identified using 'pylint --disable=all --enable=W0611'
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Tests are expected to be directly invoked when debugging so must
have execute permission.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20241217155953.3950506-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Without descriptor libvirt cannot discover the EDK II binaries via
the qemu:///system connection.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Message-ID: <20241212090059.94167-1-heinrich.schuchardt@canonical.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
With the default BIOS being compatible with amd-sev-es add the feature
to the json to indicate it's support
Signed-off-by: Pratik R. Sampat <pratikrajesh.sampat@amd.com>
Message-ID: <20241118161405.208437-1-pratikrajesh.sampat@amd.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Add new -shim command line option, wire up for the x86 loader.
When specified load shim into the new "etc/boot/shim" fw_cfg file.
Needs OVMF changes too to be actually useful.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-ID: <20240905141211.1253307-6-kraxel@redhat.com>
Add a new "etc/boot/kernel" fw_cfg file, containing the kernel without
the setup header patches. Intended use is booting in UEFI with secure
boot enabled, where the setup header patching breaks secure boot
verification.
Needs OVMF changes too to be actually useful.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-ID: <20240905141211.1253307-5-kraxel@redhat.com>
Load the complete kernel (including setup) into memory. Excluding the
setup is handled later when adding the FW_CFG_KERNEL_SIZE and
FW_CFG_KERNEL_DATA entries.
This is a preparation for the next patch which adds a new fw_cfg file
containing the complete, unpatched kernel. No functional change.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-ID: <20240905141211.1253307-4-kraxel@redhat.com>
If the binary loaded via -kernel is *not* a linux kernel (in which
case protocol == 0), do not patch the linux kernel header fields.
It's (a) pointless and (b) might break binaries by random patching
and (c) changes the binary hash which in turn breaks secure boot
verification.
Background: OVMF happily loads and runs not only linux kernels but
any efi binary via direct kernel boot.
Note: Breaking the secure boot verification is a problem for linux
kernels too, but fixed that is left for another day ...
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-ID: <20240905141211.1253307-3-kraxel@redhat.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
